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(57) Abstract 



A method and apparatus for peiforTning video 
image compression and decompression are disclosed. 
The video image compression is performed using 
boundary-spline-wavelet decomposition, in which the 
wavelets applied to sample locations ai the boundariet 
of image intervals are dlfTercni ftom those applied to 
sample locadons within the intervali. The decom- 
positioD is performed first for horizontal rows of the 
image data, and then in a vertical direction npoo the 
resuhs of the first decon^xuidotL Quantization serves 
to locally round off die higher frequency componenu 
of the decomposition, aitd the deicompoaitioo it re- 
peated until the desired conq>restion ratio is obtained. 
Lossless compression may then be applied to the de- 
composed image data, and tbe compressed image is 
transmitted or stored, depending upon the tpplicaticn. 
Decompression is efliected by lossless decompreasion 
of the received data, followed by reconstruction of 
the image using boundary-spline-wavelets, itpeated 
as necessary to fully reconstruct the image, lie re- 
constructed image can then be displayed on a con- 
ventional video display. 
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BOUNDARY-SPLINE-WAVELET COMPRESSION FOR VIDEO IMAGES 



This invention- is in the field of video image storage 
and communication, and is more specifically directed to 
compression and decompression of video images, including 
motion video images and high-quality still video images. 

5 

Background <?f the Invent io^ 

Conventional video compression 

10 

Modem electronic data processing systems are now 
often used not only for conventional numeric and text 
processing tasks, but also for processing data 
corresponding to visual images. As such, many modem 
15 computer and telecommunications systems have "multimedia" 
capabilities, where the data processed and communicated 
includes video images either generated by a computer or 
digitized from a conventional video camera. 

20 The capacity required to communicate video images on 

a real-time basis is huge, however, when measured against 
modem capabilities. For example, a single video image 
(i.e., frame) displayed by a rectangular array of picture 
elements ("pixels") arranged in 64 0 rows by 800 columns, 

25 with the color of each pixel represented by twenty- four 
digital hits, would require over 1.52 million bytes (1500 
)cbytes) of digital memory to store all information in the 
frame. While this memory requirement for a single video 
frame is large, digital storage of a series of frames, such 

30 as a motion picture, will quickly consume the disk storage 
capacity of even high-end computers and workstations. 
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The large amount of digital data necessary to 
represent a video frame not only impacts computer storage 
requirements, but limits the rate at which conventional 
systems can communicate motion pictures. Considering that 
conventional high-speed digital communication channels have 
a bandwidth of on the order of from 4 0 to 8 0 MHz, it 
becomes quite apparent that conventional motion pictures of 
thirty frames per second, with 1500 kbytes of digital 
information per frame, cannot be fully transmitted in real- 
time over state of the art digital communications systems. 

In response to these limitations on the ability of 
modem computer systems to store and communicate video 
information, various types of data compression techniques 
have been developed in recent years. Conventional data 
compression techniques are generally referred to as of 
either "lossless" or "lossy", depending upon whether data 
is discarded in the compression process. 

A survey of conventional lossless data compression is 
provided in Simon, "Lossless Compression: How it Works", PC 
Magazine (June 29, 1993), pp. 305-13. Examples of 
conventional lossless data compression techniques include 
Huffman encoding, Fano-Shannon encoding, auid arithmetic 
encoding, as well as dynamic variations of the Huffman and 
Fano-Shannon probabilistic encoding techniques . In 
general, lossless compression techniques are primarily used 
to compress entered data such as data generated from data 
processing operations, rather than sampled data 
representative of analog video or audio signals, as 
decompression will reproduce all bits of the original 
compressed data stream. 

Lossy data compression techniques, in contrast, 
provide additional data compression efficiency over 
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lossless data compression, as some amoiint of data is 
discarded in these techniques. As a result, lossy data 
compression techniques are generally used on sampled data, 
as some amount of inaccuracy relative to the true input 
5 data is tolerable; lossy data compression is, of course, 
inappropriate for use on entered data streams such as those 
produced by a data processing operation. Accordingly, lossy 
data compression techniques are widely used in the field of 
compression of video and motion picture images to obtain a 
10 high degree of compression, as some inaccuracy may be 
tolerated. A survey of conventional lossy data compression 
techniques may be found at Simon, "How Lossy Data 
Compression Shrinks Image Files", PC Magazine (July 1993), 
pp. 371 et seq. 

15 

A popular conventional lossy data compression 
technique is referred to as the JPEG (Joint Photographic 
Experts Group) method. A description of this technique may 
be found in Bamsley and Hurd, Fractal Image Compression 

20 (AK Peters, Ltd., 1993), pp. 219-228. The JPEG compression • 

method initially divides the image into blocks of pixels, 
and a Discrete Cosine Transform (DCT) is performed on each 
pixel block, producing a representation of the block as 
coefficients corresponding to frequencies and amplitudes, 

25 rather than corresponding directly to color information. 

These coefficients are then quantized, or rounded off, and 
a difference algorithm is performed over all quamtized 
blocks in the image, in a selected scan order. This 
difference algorithm subtracts a DC term corresponding to 

3 0 the mean pixel value of a block, from the DC term of the 
preceding block. The difference coefficients are then 
scanned in a different order, such as a zig-zag order, and 
the non-zero coefficients (i.e., blocks in which a 
difference from the preceding block occurred) are coded to 

35 indicate the number of preceding zero coefficients (i.e.. 
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the number of pixel blocks in which no change occurred) and 
also the value of the non-zero difference. Lossless 
compression is then often applied to the coded result to 
further compress the data. Decompression is performed by 
reversing the compression process, producing the 
displayable image. 

While the JPEG conventional video image compression 
technique is useful in obtaining high degrees of 
compression, it has been foiind that JPEG compression is 
incapable of being used in a real-time fashion for a motion 
picture. This is because the time generally required to 
perform the JPEG decompression of a motion picture frame 
exceeds the display time for the frame (1/30 second) , and 
as a result the motion picture image cannot be decompressed 
for real-time display. Temporally accurate display of a 
motion picture compressed according to these techniques, 
thus requires the decompression and display to be done in 
two steps, with the decompressed motion picture stored on 
video tape or another medium from which the motion picture 
can be played with the proper time base. 

Another conventional method of lossy video image 
compression referred to as Recursive Vector Quantization 
(RVQ) quantizes the pixel blocks directly, without a DCT or 
other transform, according to a set of selected reference 
tiles. See Simon, July 1993, op. cit . . The reference tiles 
are selected according to an iterative . technique, based 
upon the accuracy of the results relative to the original 
image. As noted in the Simon article, compression 
according to the RVQ method is computationally intense and 
complex, but decompression can be done quite rapidly. 

Another type of conventional lossy video image 
compression techniques is referred to as fractal 
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compression. As is well known in the art, a fractal is a 
mathematical image object that is self - similar , in that the 
image can be represented in terms of other pieces of the 
image. In fractal image compression, the input image is 
5 similarly divided into pixel groups, or tiles. Each tile 
is then approximated by a transformation (contractive, 
rotational, or both) of one or more other reference regions 
of the image. The compressed image thus consists of a full 
representation of the reference region, plus the 

10 transformation operators for each of the tiles. Each tile 
of the image is decompressed by performing a transformation 
of the reference region using the stored transformation 
operator for that tile. Detailed descriptions of 
conventional fractal image compression techniques and 

15 systems for performing the same may be found in Bamsley & 
Hurd, Fractal Image Compression (AK Peters, Ltd., 1993), in 
U.S. Patent No. 4,941,193, and in U.S. Patent No. 
5,065,447. 

20 

2. Freouencv and time windowing functions 

By way of further background, the field of wavelet 
analysis has recently become popular in the analysis of the 

25 time and frequency response and behavior of signals. The 
following section of this application is intended to 
provide a theoretical backgro\ind for wavelet analysis 
techniques in order to both convey the state of the art in 
wavelet analysis, and also to provide the necessary 

30 background for the person of ordinary skill in the art to 
fully appreciate the present invention. 

In the general sense, wavelet analysis is concerned 
with performing time -frequency localization of the signal 
35 to be analyzed (i.e,. the "input signal"). Time -frequency 
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localization refers to the analysis of a portion of the 
frequency spectrum of the input signal ever a selected time 
window. As will become apparent from the description in 
this specification, time -frequency localization of an input 
5 signal enables data .processing techniques to be applied to 
the signals for a wide range of purposes. 

a. Conventional analog filtering 

10 

In the time domain, frequency-windowing ie done by 
convolving a time-domain window filter function with the 
input signal; in the frequency domain, the frequency- 
windowing is done by multiplying the spectrum of the input 

15 signal with the frequency- domain transfer function of the 
filter function. Typical filter functions include low-pass 
filters (e.g., the Shannon sampling function) and band-pass 
filters. Through use of such filters, a bandwidth limited 
signal fjj^t) (i.e., zero amplitude for all frequencies 

2 0 above a limit JJ) may be decomposed into the sum of a low- 
frequency component with a series of non-2ero frequency 
bands. An expression for such a decomposition is as 
follows : 

fQ(t) =fQ^„^{t) +gQ,i(t)+. . .+gQ^^(t) 

25 

[1] 

where f ie the low-pass filtered {(i)<<t}0) component of 
the input signal fjj(t), where Sfj^itt) is the band-pass 
30 filtered signal for the ith frequency band, and where CJ^ « 
n. An ideal low-pass filtered component fjj corresponds 
to the time-domain convolution of the well-known Shannon 
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sampling function with the input signal. Ideal band-pass 
filtering may be performed by time-domain convolution of a 
filter function of the type: 

ni 

[2] 

with the time-domain input signal, and ^^^-l ^^eing the 
upper and lower limits of the frequency band.' 

Each of the low-pass and band-pass filter functions 
provide ideal frequency localization of the input signal 
fj2(t) . such that each expression fjj^^Q and j_ (t) provide 
precise information regarding the frequency spectrum of 
input signal fjjlt) within its respective frequency band. 
However, the time localization provided by this 
decomposition is quite poor, as these filters do not 
provide very precise information about when (in the time 
domain) a signal behaves differently within certain 
frequency ranges. As many important real -world signals 
include brief periods of time of rapid transient change, 
analysis of signals decomposed in this manner will not be 
able to account for the time at which such transient 
behavior occurs. For signal analysis where time 

localization is important, conventional Fourier analysis 
techniques therefore falls short of the need. 



b. Time and freouencv windnwir^o 

Certain conventional signal analysis techniques have 
addressed this problem by time -windowing the input signal. 
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thus allowing time-localization as well as frequency 
localization. According to these techniques, a window 
function h(t) is applied to the input signal f (t) to window 
the input signal near a specified point in time t=b. This 
windowing may be considered by the integral transform: 

jf[t)h(t-b) dt 

[3] 

where the tar over the function h(t-b) denotes complex 
conjugation. For a real-valued even windowing function 
h(t), this windowing process corresponds to convolution, 
allowing treatment of the windowing function h(t) as a 
filter. For example, if h(t) is the Shannon sampling 
function, the windowing process of the above equation will 
apply a low-pass filter to the input signal. However, it 
has been observed that the slow decay of the Shannon 
sampling fimction over time results in a very imprecise 
time-domain windowing operation. 

Those windowing functions h(t), for which the square 
of the magnitude have finite first and second moments and 
finite energy, (i.e., that decay sufficiently rapidly at 
infinity) will produce a time-window having a "center" 
and a "radius" h.^. The center t^ may be calculated as a 
mean value, while the radius may be calculated as the 
standard deviation of the windowing function around the 
mean t^. For a windowing function of radius L^, the width 
of the time- window will be 2A^, commonly referred to as the 
RMS duration of the windowing function h(t) . In the 
frequency domain, if the square of the magnitude of the 
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Fourier transform h{u ) of the windowing function h(t) has 
finite first and second moments, the frequency domain 
windowing function f\{u ) will have a center U)^ and a width 
2A^, calculated in a manner similar to the mean and 
standard deviation (doubled) of the frequency domain 
function Pl(a; ) ; the width 2Afj is usually called the RMS 
bandwidth of the windowing function h(t). If the RMS 
duration is finite, then the time-domain windowing 

function h(t) is a time-window; similarly, if the RMS 
bandwidth 2A^ is finite, the frequency- domain windowing 
function Pl(w ) is a frequency window. 

Referring back to the ideal low-pass and band-paes 
filter time-domain functions noted above, it will be 
readily apparent that their first moment is infinite, 
meaning that neither of these ideal filter functions can 
serve as time windows if used as windowing functions h(t) . 
However, the frequency domain representations of the ideal 
low-pass and band-pass filter functions have finite RMS 
bandwidth 2A^, so that these filters provide ideal 
frequency- localization as is evident from their ideal 
nature. 

As discussed above, accurate analysis of real-world 
signals containing transient behavior requires both time- 
localization and frequency- localization. The Uncertainty 
Principle has identified those windowing functions h that 
provide both time windowing and frequency windowing 'as 
those functions that satisfy the following inequality: 

[5] 



f 
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It has also been . previously found that the only types of 
windowing functions h(t) that achieve the lower bound of 
the Uncertainty Principle are those of the form: 



hit) =ce^'^'^e^^-"^^'/*« 



for some constants a, b, c and a with a > 0 and c *» 0. 

Further indication of the presence of a time-f recpjency 
window by any windowing function h(t) may be obtained 
through the Parseval identity. The generalized windowing 
function h(t) noted above corresponds to: 



[7] 



With reference to the left side of equation [6] , the time 
window is given by: 

[b + t* - A,, b + t' + AJ [8] 

such that the time window that is centered on t - t^ is 
shifted by the parameter b; the time window also has a 
radius of Aj^ in the time dimension. Similarly, with 
reference to the right-hand side of equation [6] , the 
frequency window is given by: 



- Ap,, ai' + Aj^] 
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and is thus centered at U)^^ with width 2A|^. Figure la 
illustrates the location of time- frequency window 2 of 
eg-jation [S] in a time -frequency coordinate system. 

For causal real -valued window functions, where: 

the function | fl(^. ) | is an even function, so that the center 
o;^ is located at U^Q and the frequency window of equation 
[9] becomes 



f-'^h' -^jV [loj 

However, while the time -frequency window of the filtering 
function h(t) may be moved along the time axis of Figure la 
by changing the value of b, the window is fixed in 
frequency at the center frequency or, in the case of 
real -valued even functions as noted above relative to 
equation [10], is. fixed at a center frequency 6;^ = o, as 
shown in Figure lb. This fixation in frequency of the 
time -frequency window in limits the usefulness of the 
windowing process of equation [3] . 



£^ — The Short -Time Fo urier Transform ffiTP^) 

Rudimentary Fourier theory indicates that translation 
in the time -domain corresponds to a phase -shift in the 
frequency domain, and conversely that a phase- shift in the 
time-domain corresponds to a translation in the frequency 
domain. Accordingly, a phase shift in the windowing 
process of equation [3] should allow for sliding of the 
frequency window along the frequency- axis . 
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Considering a real-valued window function ^(t) which 
serves as a low-pass filter (i.e., ^(4^ = 0) = i], and for 
which (pit), |t|^0(t) and t0(t) are in L^, the short-time 
Fourier transfonn of ^(t) is defined by: 

{G^f) {b,i) :=|f (t)e-^^^(t)(t-2))dt 

[11] 

for f € L^. The short -time Fourier transform (STFT) of 
equation [11] is also referred to in the literature as the 
Gabor transform. Applying the Parseval identity results in 
the following expression for the STFT, from which the 
center and radii of the time and frequency windows are 
apparent : 

[12] 

The short -time Fourier transform of equations [11] and 
[12] provide the improvement over the simple time -windowing 
process of equation [3] in that the frequency window 
function ^(w) is able to slide along the frequency axis. 
The frequency window for a windowing function ^ (w ) that 
otherwise (i.e., for ^=0) has its center at is now 

localized to frequencies near W » ^ : 

- a;, ^ + a;] [13] 
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Similarly, where the center t^ of the windowing function 
^(t) is also otherwise (b=0) located at the origin, the 
time window is now localized near t=b as follows: 

[b - A,, b + AJ [14] 

The STFT thus allows for sliding of the time- frequency 
localization windows both in time and in frequency, merely 
by changing the values of the phase-shift factors b, ^. 
respectively. Figure Ic illustrates the position in time- 
frequency space of two time- frequency windows 5^, 5^, 
having phase -shift factors [b^, ^q), (b^^, , 
respectively. As a result, the STFT allows the lowpaes 
window function to perform bandpass filtering by changing 
the value of ^. However, as is evident in Figure Ic, the 
widths of the windows are invariant with changes in time- 
shift factor b or frequency- shift factor ^. Accordingly, 
while analysis is improved through use of the short -time 
Fourier transform, inaccuracies due to undersampling have 
been observed for those transient periods of time in which 
rapid changes (i.e., amplitudes over a wide range of 
frequencies) exist. Accordingly, not only is the ability 
to slide the localization windows in both time and 
frequency desired, but it is also desirable to allow for 
scaling of the window widths as a function of time or 
frequency. 
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3. Wavelet- analvsiB 
Theor/ 

5 Wavelet analysis techniques address the need for time- 

frequency localization windows of flexible size, primarily 
by introducing a scale parameter that is mapped to the 
frequency variable. As a result, the scale parameter will 
cheinge the widths of the windows as a function of 

10 frequency, thus changing the aspect ratio of the windows 
with changes in frequency. Since the Uncertainty Principle 
requires the area of the windows to remain above a certain 
value, the time window width decreases and the frequency 
window width increases proportionally as the center 

15 frequency ^ increases. The narrowing of the time window 
and widening of the frequency window for high frequency 
environments more precisely detects and analyzes these high 
frequency portions of input signals. 

20 The basic wavelet transform is known. as the integral 

wavelet trauisform, or IWT. The IWT includes a scale 
parameter a in its definition as follows: 




25 [15] 

As such, the window function (p narrows with changing values 
of the scale parameter a, such that the time width of (ji 
decreases with decreasing a. The windowing function ^ (t) 
30 used in the IWT of equation [15] is to be real-valued as 
before, but the IWT constraints also require ^ (t) to be a 
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baridpase filter rather than a low pass filter, such that 
its Fourier transform ^(^-0) - 0, stepping at least zero 
frequency components of the signal . Since the windowing 
function ^ (t) is real-valued, its Fourier transform ipiu) 
5 satisfies: 

ifi'U)) =<If (CO) 

[1€] 

10 

so that \lp{u)\ is an even function. Because only 
nonnegative frequencies are of interest, and since ^(t) is 
a band-pass filter, the Fourier transform ljl{cj) need only 
be considered as a frequency window in the frequency domain 

15 [0, ") , with the centers and widths of the frequency window 

function ijf {(t) ) being modified as a result. For a windowing 
function V' (t) in L^, for which |t|^(t) and t^ (t) are also 
in L such that ifJ it) is real-valued, and where ^ (W=0) «= 0, 
the one-sided (i.e., nonnegative) frequency window center 

20 U) ^ as a function on the domain 10, ») is defined as; 

^^''--^ 

0 

[17] 

25 and the one-sided radius of 4f{u) is defined as: 



1 



16 





0 



J|t[^(a)) pdG) 



0 



[18] 



10 



15 



This allows the generation of an integral wavelet 
transform (IWT) using a normalization factor a'^ based upon 
the scale parameter a, which scales the time -width of the 
window as a function of frequency. For a windowing 
function Ip it) that satisfies the conditions for equations 
(17] and [18] above, the IVTT is defined as follows: 



For the IWT of equation [19] , the bandpass window- function 
^ (t) is commonly referred to as the analyzing wavelet. 

As is known in the wavelet analysis field, and given 
the foregoing discussion, it is important that the integral 
wavelet transform allows for frequency localization 

where the width of the time window is mapped to the 
frequency domain, and in which the frequency window can 
slide along the frequency axis. For finite-energy real- 
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valued input signals f(t), and since ip {t) ie real, the 
following relationship holds: 

f (-0)) e^'^'-tp(-ao)) =i^(o>) 6"^^*^$ (ao)) 

[20] 

Through the Parseval identity, one can then derive the IWT 
as follows: 

0 3 

[213 

for all f e L^j^, where W ^* is the one-sided center of lp{a)) 
on the domain [0, ") , and where rj is defined as follows: 

Ti (co) : =^ (o)+(o!) 

[22] 

As noted above, it is desirable to map the scale 
parameter a to the frequency at which the time- frequency 
window ie to be localized. Accordingly, the scale 
parameter a is mapped to the shift frequency parameter ^ as 
follows: 
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for some c > 0, where c is a- calibration constant. 
Substituting for the scale parameter a defines the IWT as 
follows : 

[24] 

It is convenient to set c=&>^*, so that: 

iwlf) (jb, 5) =^i?e?f (0)) e-^^r\ {a (w-^ ) do 

[25] 

This produces a frequency window rfiaiU-^)) that slides 
along the frequency axis with the value of ^ , having a 
range : 

[26] 
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The width of this- window thus increases at higher 
frequencies ^, as reflected in smaller values of the scale 
parameter a. In terms of the scale parameter a, the width 
of the frequency-window is as follows: 

(o):-2^;) ,1 (a)>A;)] 

127] 

where the frequency shifting term ^ = w */a, such that the 
frequency-width of the frequency window increases with 
increasing frequency ^ (decreasing values of a) . Along the 
time-axis, the time window of the IWT of equation [19] is 
given by: 

[28] 

As a result, the width of this time-window is 2aA^, which 
decreases at higher frequencies ^ (and lower values of a) , 
and which increases at lower frequencies ^ . For the 
transform of equation [15] , Figure Id illustrates three 
time -frequency windows 6q, 6^, 6^, with varying translation 
factor pairs (bp, ^ q) , (b^, ^^). {h^. ' respectively. 
As is evident in Figure Id, both the time-width and 
frequency- width of windows 6 vary with varying shift 
frequency ^, such that the time-width decreases and the 
frequency-width increases with increasing ^ . 
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Accordingly, it should now be apparent to those in the 
art that wavelet analysis is capable of providing increased, 
accuracy analysis of signals, particularly those including 
transient components, considering that, for higher 
5 frequency analysis, the width of the time window decreases 
and the width of the frequency window increases. This 
ensures adequate sampling of the input signal function, and 
also allows for determination of the exact time in the 
signal at which the transient event occurred. 

10 

As is known in the art of wavelet analysis, however, 
the definition of the proper wavelet function (p (t) is of 
great importance. Various specified analyzing functions 
have been used in wavelet analysis, with the selection of 
15 the function made according to computability, or according 
to attributes of the signal under analysis. 

b, Prior wavelets 

20 Wavelet signal analysis has been applied to signals 

produced in seismic exploration for oil and gas, as 
described in U.S. Patent No. 4,599,567. This reference 
describes a wavelet analysis technique using Morlet's 
wavelet as the analyzing wavelet. Morlet's wavelet is a 

25 sinusoid limited by a Gaussian probability envelope to a 
finite duration; the envelope may or may not be skewed 
toward either the leading or lagging edge of the time- 
domain envelope of the wavelet, as desired for the type of 
signal under analysis. This reference also discloses 

30 circuitry for performing the wavelet analysis of an 
incoming signal using such analyzing wavelet. 

Another finite duration analyzing wavelet was proposed 
by Yves Meyer. This analyzing wavelet is substantially a 
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finite duration uniform magnitude level over the wavelet 
window, analogous to the Shannon sampling function. 

Other wavelets have been proposed which are not 
expressible by a mathematical formula, but instead are 
utilized as numeric wavelets. These wavelets include the 
Battle -Lemannfe wavelet, which is a spline function of 
infinite duration, and the Daubechies wavelet, which is a 
fractal wavelet of finite duration. The lack of explicit 
formulae for these wavelets limit their applicability for 
rapid and accurate computer implementation of the wavelet 
analysis technique in computer hardware. 

Another previously published wavelet is the Chui-Wang 
wavelet, which is a wavelet of finite duration but which 
may be expressed in an explicit formula. 

The support for each of the prior wavelets noted above 
is over an unbounded interval. However, since real -world 
problems require the application of the wavelets to bounded 
intervals, wavelet analysis of input signals using these 
prior wavelets result in errors of the type commonly 
referred to as "boundary effects". 

Figure 2 graphically illustrates the reason for 
boimdary effects arising from such conventional wavelets. 
Conventional wavelet 7 illustrated in Figure 2a is first - 
order spline-wavelet that is based on a function (jl having 
moments with the following properties (for i = 0, 1, . . . tn- 
1, and some m as 1) : 




[293 
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where a, b are scaling parameters as discussed above. The 
conventional wavelet 7 of Figure 2a is not orthogonal over 
a bounded interval [c, d] , however, meaning that: 



for all integers i > 0. 

Figure 2 graphically illustrates the performance of 
the IWT using wavelet 7 having the above noted properties 
at a point in the time series which happens to be at or 
near the boundary of an interval [c, d] . Data points f^, 
^3' ^4 correspond to input signal samples within the 
interval, which are plotted against wavelet 7 in Figure 2 ; 
in this example, for purposes of explanation, the input 
signal sample data closely matches the shape of wavelet 7 
within the interval [c, d] . The position of wavelet 7 
corresponds to the position, in performing the IWT, of the 
sample point of interest at the boundary value t=a. Since 
wavelet 7 at this position requires support outside of the 
interval [c, d] for which input signal data exists, the 
zero values that must be assumed for the data points f^, f^ 
outside of interval [c, d] necessarily fail to match 
wavelet 7. This will result in an non-zero result for the 
IWT, even where the input data signal within the interval 
[c, d] exactly matches wavelet 7. As is well known in the 
field of signal processing, this inaccuracy due to wavelet 
7 requiring support outside of the bounded interval [c, d] 
is made manifest by boundary effects at the edges of the 
bounded interval, since the unbounded wavelet 7 fails to 
accurately represent the series of actual input signal 
sample data. 



d 




c 
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In the field of video image compression and analysis, 
boundary effects greatly affects the quality of the image 
displayed after compression and decompression. This is 
because the boundary effects will appear as false image 
data at the edges of pixel blocks corresponding to bounded 
intervals, and also at the edges of the image (even if no 
subdivision of the image is performed) . The inaccuracies 
due to boundary effects also limit the ability to magnify 
an image when displayed, considering that the magnification 
will make the boundary effect errors to become even more 
visible. 



The boundary- gplinA - wave 1P^ 

By way of further background, a bounded interval 
wavelet is described in Chui and Quak, "Wavelets on a 
Bounded interval-, Numerica] Methods nf ^ ppro^im;,^Sor^ 
Theoii:, Volume 9 (Dec. 1992), pp. 53-75, incorporated 
herein by this reference. This wavelet, which has an 
explicit formula, is not only a function of the time 
variable t. but is also a function of the position of the 
sample within the internal [c, d] , so as to account for 
boundary effects. In effect, sample locations near the 
boundaries of the interval will correspond to different 
wavelet shapes than will sample locations within the 
interval that are away from the boundaries. Boundary 
effects are eliminated as the boundary wavelets do not 
require support outside of the interval. 



Referring now to Figures 3a through 3d, an exemplary 
set of four first -order wavelets 8 according to the Chui- 
Ouak boundary-spline-wavelet approach are illustrated. 
Figure 3a illustrates the shape of "boundary" wavelet 8 
for a sanple location near the boundary t=a of the interval 
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(a, b] , while Figure 3d illustrates the shape of boundary 
wavelet 8^ for a sample location near the boundary t=b of 
the interval [a, b] . Figures 3b and 3c each illustrate the 
shape of "inner" wavelets 8^ for sample locations within 
the interval [a, b] away from the boundaries. As is 
evident from Figures 3a through 3d, boundary wavelets 8^, 
8^, have different shapes than inner wavelets 8^ {which 
have the same shape as one other) . As is further evident 
from Figures 3a through 3d, neither inner wavelets 8^ nor 
boundary wavelets 8^, 8^^ require support outside of the 
interval [a, b] , or: 

b 

a 

[31] 

for i = 0, 1, . . m-1, and for some m a 0, for the 
entire set of wavelets 8 of Figures .3a through 3d. 
Accordingly, application of the set of boundary- spline- 
wavelets 8 to actual real -world data, for which the time 
interval is necessarily boiinded, will not produce boundary 
effect artifacts. 

Other boundary-wavelets are known, as described in 
Davibechies, "Two recent results on wavelets : Wavelet bases 
for the interval and biorthogonal wavelet diagonal i2ing the 
derivative operator", Recent Advances in Wavelet Analvsis . 
Schumaker and Webb, ed. (Academic Press, 1993), pp. 237-58. 
These wavelets are not spline functions, and do not have 
explicit formulae. As a result, it is believed that these 
wavelets are of limited effectiveness in video image 
compression and decompression. 
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4. Ob-iert q of the invention 

It is therefore an object of the present invention to 
apply wavelet analysis tools to the task of video image 
compression for storage and transmission. 

It is a further object of the present invention to 
provide an apparatus for perforTiiing video image compression 
according to a selected wavelet. 

It is a further object of the present invention to 
provide an apparatus for receiving a compressed video image 
and to decompress the same for real-time playback of the 
stored or communicated video image information. 

It is a further object of the present invention to 
provide such a method of compression and decompression such 
that the compressed image may be played locally at real- 
time . 

It is a further object of the present invention to 
provide such a system and method which allows for high 
accuracy magnification of the decompressed image, with much 
reduced incidence of edge effects. 

It is a further object of the present invention to 
provide such a system and method which provides a high 
degree of compression. 

It is a further object of the present invention to 
provide such a system and method which can utilize dynamic 
compression on a frame -by- frame basis, such that high 
frequency frames may be compressed to different ratios than 
low frequency frames. 
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It is a further object of the present invention to 
provide such a system and method which facilitates 
interactive display of a motion image, including insertion, 
editing and repetitive display. 

It is a further object of the present invention to 
provide such a system and method which provides the ability 
for slow display systems to skip certain frames so that a 
motion picture can be displayed on a real-time basis by 
such slower systems, although with fewer frames per second. 

It is a further object of the present invention to 
provide such a method and system which allows for division 
of an image into several portions for purposes of 
compression and communication or storage, with subsequent 
display of the full image after decompression. 

It is a further object • of the present invention to 
provide such a method and system which allows for higher 
lossy compression ratios by further quantization, as useful 
in compressing and decompressing high-quality 24 -bit still 
images. 

Other objects and advantages will be apparent to those 
of ordinary skill in the art having reference to the 
following specification, together with its drawings. 
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gummarv of the Invention 

The present invention may be implemented into a method 
and apparatus for compressing and decompressing video 
images. The compression/decompression system receives 
video image data in a suitable format, such as eight-bit 
Portable Grey Level format, and includes circuitry for 
decomposing each component of the image into low and high 
frequency components by way of boundary- spline-wavelet 
decomposition. The system also includes circuitry, for 
quantizing the high frequency portions of the 
decomposition. The decomposition may be repeated tmtil the 
desired compression ratio is obtained. Lossless 
compression may then be performed on the compressed image, 
following which the compressed data may be stored for 
archival purposes, or may be transmitted to a receiving 
station. 

According to the present invention, in decompression 
mode, the system performs lossless decompression of the 
stored or communicated image data, according to the 
compression scheme utilized. Reconstruction of the image 
is then performed by way of the boundary- spline -wavelet 
approach, in a reverse fashion relative to the image 
compression. The result of the decompression is then 
formatted as desired for display. 
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Brief DescrlDfinn of the Drawinog 

Figures la through Id are time -frequency plots 
illustrating the concept of time -frequency windowing, as 
described in the background of the invention. 

Figure 2 is a graphical illustration of a wavelet with 
non-zero moments of amy order relative to the interval [c, 
d] , as applied to an input signal, and which is the source 
of boundary effects. 

Figures 3a through 3d are graphical illustrations of 
an exemplary set of boundary- spline -wavelets as used in the 
preferred embodiment of the invention. 

Figure 4a is an electrical diagram, in block form, of 
a system for producing and communicating or storing 
compressed video image information according to the 
preferred embodiment of the invention. 

Figure 4b is an electrical diagram, in block form, of 
the image format converter in the system of Figure 4a. 

Figure 5 is an electrical diagram, in block form, of 
a compressor according to the preferred embodiment of the 
invention. 

Figure 6 is a flow diagram illustrating a method of 
compressing video image data according to the preferred 
embodiment of the invention. 

Figure 7 is a detailed flow diagram illustrating a 
method of performing the boundary-spline-wavelet 
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decompression according to the preferred embodiment of the 
invention. 

Figure 8 is a series of frequency domain plots 
5 illustrating the decomposition of the video image according 
to the preferred embodiment of the invention. 

Figures 9a through 9c are matrices illustrating the 
effect of decomposition of the video image according to the 
10 preferred embodiment of the invention. 

Figure 10 is a flow diagram of a preferred matrix 
algebra technique for performing the boundary- spline- 
wavelet decomposition of Figure 7. 

15 

Figure 11 is a chart illustrating the relationship of 
the matrices in the process of Figure 10. 

Figures 12a through 12e illustrate an example of a 
2 0 video image decomposed according to the preferred 
embodiment of the invention. 

Figures 13a through 13c illustrate the quantization of 
the decomposed video image of Figures 12a through 12e 
2 5 according to the preferred embodiment of the invention. 

Figure 14 is a diagram of the construction of a 
compressed video image frame for transmission according to 
the preferred embodiment of the invention. 

30 

Figure 15a is an electrical diagram, in block form, of 
a decompressor according to the preferred embodiment of the 
invention. 



wo 9a/19«« 



PCIAJS9M)05« 



30 

Figure. 15b is an electrical diagram, in block form, of 
a format converter useful in the decompressor of Ficrure 
15a. 

5 Figure 16 is a flow chart .of a process for 

decompressing video image data according to the preferred 
embodiment of the invention. 

Figure 17 is a detailed flow chart of a process for 
10 reconstructing video image data that were decomposed 
according to the boundary- spline-wavelet approach of the 
preferred embodiment of the invention. 

Figure 18 is a chart illustrating the relationship of 
15 the matrices as used in a preferred matrix algebra routine 
for the reconstruction of Figure 17. 

Figure 19 is an electrical diagram, in block form, of 
a decompressor system according to an alternative 
20 embodiment of the invention. 
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Detailed Descri ption of the Preferred Pmhcdiment 

1. Video image data communication system 

Referring now to Figure 4a, the construction of a 
video communication system 8 according to the preferred 
embodiment of the invention will now be described in 
detail. In this embodiment, system 8 is specifically 
designed for communicating compressed motion picture data; 
it is of course contemplated that system 8 may 
alternatively be used to communicate compressed data 
representative of a series of still video images. In 
addition, as will be described in further detail 
hereinbelow, system 8 may alternatively or also be used for 
archival storage of motion picture or still video image 
data, and the subsequent retrieval and display thereof. 

As shown in Figure 4a, in the context of video data 
communication, the transmission end of system 8 includes 
video source 12 and compression system 10, while the 
receiving end of system 8 includes decompression system 20 
and video display 26. As illustrated in Figure 4a, video 
source 12 may include digital video source 12d which may be 
a conventional device such as a CD-ROM drive, scanner, 
digital electronic network connection, or similar unit, or 
alternatively may be a computer storage unit such as a disk 
that contains digital video information. Video source 12 
may also or instead include analog video source 12a, which 
may be a video camera, VCR unit, television broadcast or 
cable receiver, or another conventional source of analog 
video information. In any case, video source 12 provides 
digital or analog signals indicative of the images to be 
communicated or stored by the system of Figure 4a. 
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Communication network system 15 is a conventional 
analog transmission or electronic digital communications 
network, or both an analog and digital network when 
including the appropriate clusters, analog- to-digital and 
digital-to-analog converters, and other necessary 
apparatus. Network 15 may be realized according to any 
conventional technology, including hard- wired cable, fiber 
optic cable, broadcast or satellite transmission, and the 
like. It is further contemplated that network 15 may be 
implemented merely by the physical transportation of 
portable media 22' such as floppy diskettes, CD-ROMs and 
the like. Regardless of the implementation, network 15 is 
connected between compression system 10 and the input of 
decompression system 20 to communicate compressed video 
image data therebetween. 

It is specifically contemplated that the video data 
transmission effected from compression system 10 may be of 
the broadcast type, such that a single transmission end 

(source 12 plus compression system 10) may communicate 
simultaneously or in secfuence to multiple receiving ends 

(decompression system 20 and display 26) . For example, 
compression system 10 may be located at a television or 
movie studio, or at a local cable television system "head- 
end", with multiple receiving ends (decompression system 20 
and display 26} located at homes, offices, or local 
theaters, depending upon the particular transmission being 
effected. 

Figure 4a also illustrates the optional use of system 
8 in the context of the archival storage of motion picture 
or still video image data, and its later retrieval and 
display thereof. Disk storage unit 22 is coupled to 
receive the compressed video data from compression system 
10 for storage. An exan^jle of disk storage unit 22 is a 
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large disk (e.g., having capacity on the order of 1000 
Gigabytes) ; alternatively, or in addition to the large disk 
unit, disk storage unit 22 may be associated with a 
mainframe or supercomputer to provide services such as on- 
line collection of video images into a library form. Disk 
storage unit is also coupled to decompression system 20, 
which is operable in this context to receive compressed 
video data therefrom for display on video display 26. The 
subsequent retrieval of the archived information in disk 8 
may be made via a digital communications network (such as 
network 15 described hereinabove), or alternatively by way 
of a portable data storage medium such as a tape or 
diskette. 

Compression system 10 includes data format converter 
14 that is functionally connected between video source 12 
and compressor/coder 16. Format converter 14 is of 
conventional construction as used to convert the format of 
the video image data from video source 12 into the format 
suitable for compression by compressor 16. In the example 
described hereinbelow, the format utilized by compressor 16 
is eight -bit Portable Grey Level (PGM) , although fewer than 
eight bits may be used in the PGM format, depending upon 
the available hardware devices and architecture. For color 
images, the format is RGB- PGM, where each of the red, green 
and blue color components is separately stored and 
commiinicated in PGM format. The PGM image format will be 
described in further detail hereinbelow. Accordingly, 
format converter 14 converts the image data from video 
formats such as PCX, IMG, GIF, TIF, RLE, NTSC, PAL, and the 
like into PGM or RGB-PGM format. Of course, if the output 
from video source 12 is already in PGM or RGB-PGM format, 
format converter 14 is unnecessary, and compressor 16 may 
directly receive the data from video soiirce 12. 



8^lffB1fffflER<RBKi26)i 04/01/2005 



wo 9S/mS3 



PCT/DS9M)e«3 



34 

Referring now to. Figure 4b, the construction of format 
converter 14 according to the preferred embodiment of the 
invention will be described in detail, for the example 
where the output provided by format converter 14 is in RGB- 
5 FGM format. As noted above, format converter 14 can 
receive video data from either analog or digital video 
sources 12a, 12d, respectively, and as such two input ports 
are provided to format converter 14 along with two 
functional devices to reformat the received data into the 
10 desired format. 

In the example of Figure 4b, digital video image data 
presented by digital video source 12d is received by format 
decoder and color expansion device 21. The format decoder 

15 portion of device 21 decodes the data from the format 
presented by source 12d, such format being PCX, IMG, TIF, 
GIF, RLE, YUV, etc., into a color matrix, or color palette 
table, representation of the image signal. Format decoder 
circuitry is well known in the art, such as described in 

20 Rimmer, Supercharged Bitmapped Graphics (Windcrest/McGraw 
Hill) . This decoding is followed by a color expansion 
operation performed by device 21, such that the output 
therefrom is in raw RGB format, with each pixel represented 
by successive bytes of red, green and blue color 

25 components. Conventional circuitry for performing such 
color expansion is readily available in the field, 
including such devices as color palette RAMs. The output 
of format decoder and color expansion device 21 is 
presented to color separator 25. 

30 

On the analog side, analog video information is 
presented by analog video source 12a to analog screen 
grabber and digitizer device 23 . Device 23 first converts 
the analog input data from its NTSC or PAL format, by the 
35 screen grabber capturing the input screen data. Device 23 
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then digitizes each pixel of the captured screen, in the 
conventional manner. Conventional circuitry for performing. 
Che screen grabbing and digitizing functions may be used to 
implement device 23. The output of device 23 is also in 
the raw digital, or binary, RGB format as presented by 
device 21. 

Color separator 25 is a conventional digital filter 
for separating the raw binary RGB signal presented by 
either of devices 21, 23 into RGB-PGM format. As is well 
known in the art, the RGB-PGM video data format includes, 
for each pixel, three eight -bit components. These three 
eight -bit components correspond to the intensity of the 
red, blue and green colors, respectively, to be displayed 
for that pixel. Of course, other byte widths may be used 
to digitally represent the intensity of each color 
component . 

The RGE-PGM format used in this preferred embodiment 
of the invention decomposes the input video image into 
three screen representations, one for each of the red, blue 
and green color components. Each pixel location in the 
input image is thus represented by a pixel value in each of 
the red, blue and green decompositions, indicating the 
intensity of that color component at that pixel location of 
the image. In the eight -bit PGM format, the intensity of 
each color component for each pixel can thus range from 0 
(black, or no intensity) to 255 (full intensity) . The 
eight-bit PGM formaC is particularly convenient for 
processing by microprocessors or computers using ASCII 
coding and programming environments, as the fundamental 
storage unit is the byte in these environments. 

The three channels of RGB-PGM data produced by color 
separator 25 is then presented to compressor 16, either 
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sequentially or in parallel, for ccmpression as will be 
described hereinbelow. According to this embodiment of the 
invention, each of the R, G, B components are compressed, 
decompressed, and otherwise processed separately from the 
other components for that image . 

Of course, if the data presented by video source 12, 
specifically digital video source 12d, is already in RGB- 
PGM format, format converter 14 is not necessary in the 
system of Fig\ire 4a. 

Compressor 16 includes the necessary compression 
circuitry, an example of which is described in detail 
hereinbelow, for compressing the formatted digital data 
according to the boundary- spline -wavelet technique that is 
also described in detail hereinbelow. Compressor 16 may 
also include coding circuitry for formatting the compressed 
data into the suitable format for communication over 
network 15 (or storage in disk 22) . As will also be 
described in further detail hereinbelow, specific 
information regarding attributes of the communicated or 
stored information may be specifically included in the data 
coded by compressor 16. 

Similarly, decompression system 20 includes 
decompressor 18 coupled to network 15 (or to disk 22, as 
the case may be) . Decompressor 18 receives the transmitted 
or stored image data, reformats it into a form suitable for 
decompression, if necessary, and decompresses the data. 
Decompressor 18 in this embodiment of the invention 
communicates the decompressed image data to format 
converter 24, which converts the decompressed data to the 
appropriate f onnat for display by display 26 . 
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As illustrated in Figure 4a, display 2G may be 
implemented as a digital display 26d. to which digital data 
may be directly applied thereto; alternatively, display 26 
may be implemented as a conventional analog display, with 
the appropriate NTSC or other analog video format data 
applied thereto. According to this example, the output 
data from decompressor 18 is in PGM or RGB-PGM format, and 
thus format converter • 24 will convert the PGM data into 
PCX, IMG, GIF, TIF, RLE, NTSC, PAL, RGB, or another display 
format. Of course, if digital display 26d is used and is 
capable of receiving and directly displaying PGM format 
data, format converter 24 is unnecessary. 



2^ Construc tion of the comnresBriT- 

Ref erring now to Figure 5, the construction of 
compressor 16 according to the preferred embodiment of the 
invention will now be described in detail. It ia. of 
course, contemplated that other architectural arrangements 
of circuitry may be used in the compression of video data 
according to the present invention. Specifically, it is 
contemplated that a conventional general purpose computer 
system may be capable of performing the compression of the 
video image data according to the present invention. 
However, the example of Figure 5 incorporates a preferred 
embodiment of the circuitry for performing the data 
compression functions to be described herein. 

In this example, compressor 16 preferably includes one 

main RGB-PGM channel controller 28 and three substantially 

identical channel compressor subsystems 29„, 29„, 25 

K G B ' 

Three channel compressor subsystems 29j^, 29^, 29g are 
provided according to this embodiment of the invention, 
considering the separation of the input video data into the 
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three component channels by format converter 14 described 
hereinabove. Compressor 16 also preferably includes data 
flow interface 39, for receiving data from each of channel 
compressor subsystems 29 and presenting the same to network 
15. 

Main controller 28 receives the three channels of RGB- 
PGM data from format converter 14, and forwards each 
channel of data separately to the. appropriate channel 
compressor subsystem 29j^, 29^, 29g. Alternatively, 
compressor 16 may be implemented to have only a single 
channel compressor subsystems 29 which processes each 
channel of data sequentially, in which case main controller 
2 8 would control the sequential forwarding of image data to 
the single compressor subsystem 29. 

Main controller 28 is connected by way of the 
appropriate bidirectional buses and control lines to 
control the functions within compressor 16. In addition to 
its control of subsystems 29, main controller 28 also 
controls the timing, feedback and sending operation of 
compressor 16, including control of the data flow interface 
39. As such, main controller 28 is preferably a general 
purpose programmable microprocessor* or other central 
processing unit (CPU) of sufficient computational power and 
capacity to process some or all of the image data and to 
control the performing of the image compression functions 
to be described hereinbelow. It is contemplated that 
microprocessors having performance levels similar to or 
greater than those of the 80486 type (including those 
available from Intel Corporation or Cyrix Corporation) , of 
the 68040 type (including those available from Motorola) , 
and of the SPARC processor type (available from Texas 
Instruments Incorporated or Sun Microsystems, Inc.) will be 
suitable for use as main controller 28 in compressor 16. 
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The ccnetruction' of channel compressor subsystems 29 
will now be described in detail relative to the 
construction of subsystem 29j^ as shown in Figure 5; it is 
contemplated that the other subsystems 29„, 29„ will be 

K G 

similarly constructed. Each channel compressor subsystem 
29 according to this embodiment of the invention is 
specifically designed to perform the functions of boundary 
spline wavelet decomposition, quantization and lossless 
compression, as used in the compression operation according 
to this embodiment of the invention. 

Channel compressor subsystem 29 according to this 
embodiment of the invention includes the circuit functions 
of digital matrix process 30, timing circuit 37, 
quantization processor 32, lossless compressor 34, program 
data memory 35, and multiple memory banks 36 for storage of 
image data. 

Timing circuit 37 performs the functions of receiving 
the PGM format channel data from main controller 2 8 and 
forwarding the received data to memory banks 36 in its 
subsystem 2S . In addition, timing circuit 37 controls the 
timing and feedback among the other functional components 
of channel compressor siibsystem 29, including the matrix 
operations performed by digital matrix processor 30, 
quantization performed by quantization processor 32, 
lossless compression performed by lossless compressor 34, 
and accesses to memory 35, 36. 

Digital matrix processor 30 is a processing circuit of 
conventional architecture that is specifically suitable for 
performing vector and matrix operations as used in the 
decomposition of image data according to the preferred 
embodiment of the invention, as will be described in detail 
hereinbelow. As will be apparent from the- description 
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below, these operations include the retrieval of pre- 
calculated matrices from program data memory 35, the matrix 
operations utilized in performing boundary-spline-wavelet 
decomposition of the video channel data, and storage of the 
results in memory banks 35. Examples of presently 
available components suitable for use as digital matrix 
processor 30 include digital signal processors such as the 
i860 processor available from Intel Corporation and the 
TMSC4 0 digital signal processor available from Texas 
Instruments Incorporated, and also general purpose 
microprocessors such as those of the 80386 and 80486 
families available from Intel Corporation, and of the 68030 
and 68040 families available from Motorola. 

Quantization processor 32 is a logic circuit for 
filtering the data corresponding to decomposed images in 
order to achieve the desired compression ratio. It is 
contemplated that conventional processing circuitry or 
custom logic circuitry for performing this function in the 
manner described hereinbelow, will be readily apparent to 
one of ordinary skill in the art. According to' this 
preferred embodiment of the invention, quantization 
processor 32 may be selectably controlled to perform the 
quantization process according to various selectable modes. 
These modes are selectable by way of quantization mode 
register 31, which stores a three digit code defining the 
type of quantization to be performed by quantization 
processor 32. An example of the codes storable in 
quantization mode register 31 and their corresponding 
quantization modes are as follows: 
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No quantization 
thresholding 
scalar quantization 

JPEG quantization (i.e., using tables) 
reserved for other quantization modes 
(e.g., vector quantization) 

Lossless compressor 34 may be implemented by way of a 
conventional digital signal processor such as the TMSC4 0, 
programmed in such a manner as to perform lossless 
decompression upon the results of the quantized output from 
quantization processor 32. The lossless decompression 
performed by lossless decompressor 34 is according to the 
desired conventional technique, such as Huffman encoding, 
adaptive Huffman encoding, arithmetic encoding, LSQ coding, 
and the like. Alternatively, lossless compressor 34 may be 
implemented as a custom logic circuit for providing this 
function. The output of lossless compressor 34 is 
preferably compressed data for the channel (R, G, B) in 
bitstream format, for application to data flow interface 
39. 

Data flow interface 39 provides an interface between 
compressor 16 and network 15, and as such must gather the 
bitstream output from lossless compressors 34 in each of 
the subsystems 29 and arrange the same into a suitable 
format for transmission. Data flow interface 39 also 
provides a feedback signal to main controller 28 upon 
transmission of a frame of compressed data, based upon 
which main controller 28 may commence the processing of the 
next image frame to be compressed. 

The preferred format in which the output compressed 
data from data flow interface 39 is communicated over 
network 15 will be described in further detail hereinbelow. 
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This example of compreesor 16 is intended to support 
the compression of high definition real-time true color 
video image data, where "true color" indicates the use of 
twenty- four bits of color information for each pixel, 
resulting in 16.7 million possible colors. The frame rate 
for compressor 16 is intended to be on the order of thirty 
frames per second, so as to support "real-time" video image 
compression. 

As noted above, if the color and frame rate 
requirements are reduced from real-time true color video, 
it may be possible to implement compressor 16 as a single 
channel, i.e. with a single channel compression su±i8yBtein 
29. In this implementation, color data could be compressed 
sequentially for the R, G and B components of the RGB-PGM 
input data, under the control of main controller 28. 

3. Bou ndarv-S Dline-wavelet video image data compression 

Referring now to Figure 6, a method of compressing 
video image data according to the preferred embodiment of 
the invention will now be described in detail. It is 
contemplated that the architecture of compressor 16 of 
Figure 5 and described hereinabove is particularly suitable 
for the performing of this method, although it is further 
contemplated that other computer architectures and 
arrangements may alternatively be used to perform the 
process of Figure 6. 

The flow chart of Figure 6 corresponds to the 
compression of a single frame of video image data. 
Accordingly, for the compression of a motion picture, the 
process of Figure 6 is performed sequentially for each 
frame of the motion picture. In the case of still image 
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compreseion, of course, the process of Figure 6 is 
performed for each image. 

The process of Figure 6 begins with the conversion of 
the video image data for the frame into the desired format 
for compression which, according to this preferred 
embodiment, is the well-known PGM format. As discussed 
above relative to Figure 4a, this conversion is preferably 
performed by format converter 14, allowing compressor 16 to 
be dedicated to the compression process. As noted above, 
the PGM (Portable Grey Level) format expresses each picture 
element ("pixel") as a numerical value corresponding to its 
brightness;, for eight-bit PGM, the values range from 0 to 
255. Color images may be expressed in a mode referred to 
in the art as RGB- PGM, where a PGM image is provided for 
each of the red, green and blue color components of the 
image. It is believed that the RGB-PGM format is the most 
adaptable format for processing of color image data by way 
of the present invention, as it directly provides "true" 
color display information for high performance display 
systems, and is also readily adaptable for conversion to 
color look-up table ("color palette") display systems. 

After conversion by format converter 14, the RGB-PGM 
image data is separated into separate R, G, B chaimels by 
main controller 28. This allows each of channel 
compression subsystems 29 to compress the image data in the 
manner described hereinbelow in parallel with one another. 
Of course, if a monochrome image is being compressed, only 
a single subsystem 29 is necessary; alternatively, if the 
compression rate allows, a color image may be compressed by 
sequentially compressing the separate R, G, B channel image 
data. 
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^ Boundary- spline- wavelet decomposition 

After channel separation by main controller 28, a 
frame of PGM image data is stored in memory banks 36. in 
subsystem 2 9 in a row-based order, arranged from top left 
to bottom right of the image. Each PGM image frame is then 
processed by boundary- spline-wavelet decomposition, 
indicated by process 40 of Figure 6. Figure 7 is a 
detailed flow chart of process 40 according to the 
preferred embodiment of the invention, to which attention 
is now directed. The compression method of Figure 7 will 
be described relative to a single channel (R, G, or B) , as 
it will be understood by those of ordinary skill in the art 
that a full color image will be compressed by the parallel 
or sequential compression of the other color channels in 
similar fashion. 

The boundary- spline- wavelet decomposition of process 
4 0 may operates upon individual blocks of pixels of the PGM 
frame, such as blocks of pixels that may be as small as 
eight-by-eight pixels, or as large as on the order of 1024- 
by-1024 pixels or greater. Process 4 8 indicates that the 
operation of each subsystem 29 will be performed upon the 
PGM frame in subdivided blocks. One of the important 
benefits of the present invention is that the size of the 
blocks defined in process 4 8 depends primarily upon the 
architecture of subsystems 29 in compressor 16 and of 
digital matrix processor 30 therein, but is not dictated by 
considerations of picture quality. This is because the 
boundary- spline wavelets used in the decomposition 
eliminate inaccuracies in the displayed data resulting from 
boundary effects. 

Alternatively, if the computing capacity of digital 
matrix processor 30 and sxibsystems 29 is adequate, an 
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entire frame may be decomposed in process 4 0 without 
dividing the image into smaller blocks of pixels. The 
present Invention will still provide important benefits due 
to the elimination of boundary effects and inaccuracies 
5 around the edge of the image that would otherwise be 
present if unbounded wavelets were used. In addition, 
regardless of whether the image is divided into smaller 
blocks, the elimination of boundary effects according to 
the present invention also enables magnification of the 
10 image to be performed after decompression, without the 
limitations and artifacts that would otherwise be present. 

Following division of the frame into the appropriate 
image blocks, the decomposition of process 40 continues 

15 with processes 50, 52 in which boundary-spline-wavelet 
decomposition is performed upon the image block, according 
to this embodiment of the invention. As will be described 
in detail herein, according to the preferred .embodiment of 
the invention, the decomposition of process 4 0 is performed 

20 utilizing the wavelets described in the Chui and Quak paper 
cited hereinabove and incorporated by reference hereinto, 
and illustrated in Figures 3a through 3d discussed above. 
Process 50 first decomposes each horizontal row of the 
image block into two equal numbers of components of low- 

25 frequency (low-pass) and high-frequency (band-pass) , and is 
followed by process 52 in which each vertical column of 
each of the low-frequency and high-frequency results of 
process 50 are again decomposed into two equal numbers of 
components of low- frequency and high-frequency. 

30 

Referring now to Figure 8 in combination with Figures 
9a through 9c, decomposition processes 50, 52 will now be 
described in further detail. Figure 8 illustrates a 
frequency domain representation of function fj^, and which 
35 corresponds to image block 51 of Figure 9a that represents 
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the discrete function f^{x,y) . The discrete function 
fj^(x,y) is a functional representation of image block 51 in 
PGM format, such that each value "x" in Figure 9a is a 
digital value corresponding to image brightness at a 
5 position (x,y) . In this example, image block 51 is eight 
pixels square. 

As shown in Figure 8, process 50 of Figure 7 
decomposes the function fj^ into a low-frequency or low-pass 

10 component fjj-1 and a high-frequency or band-pass component 
gj^-1. According to the preferred embodiment of the 
invention, process 50 performs such decomposition for each 
horizontal row of image block 51, so that the result of 
process 50 ie a matrix 53 that includes, for each row, a 

15 low frequency portion fjj_^(x,y) and a high frequency 
portion gj^.^^^^'Y)- I" other words, the decomposition of 
process 5 0 is performed considering the image data for each 
row of image block 51 as a one-dimensional spatial function 
in the x-dimension. 

20 

As discussed hereinabove, the decomposition of process 
5 0 is performed according to" boundary- spline- wavelet 
decomposition techniques, so that boundary effects are 
eliminated in the resulting frame. Accordingly, for each 

25 horizontal row of image block 51, the low frequency 
component (or fjj_^ (x,y) in the spatial domain) in the 
corresponding row of decomposed image block 53 is a spline 
interpolation of the spatial data of image block 51, The 
high frequency component (or S^^^iXiy) in the spatial 

30 domain) in the corresponding row of decomposed image block 
53 corresponds to the boundary wavelets applied to the 
spatial data of that row of image block 51. 

Referring back to Figure 5, according to the preferred 
35 embodiment of the invention, it is contemplated that the 
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decomposition of process 50 is performed by digital matrix 
processor 30 by way of a matrix algebra technique. 
According to this matrix technique, the decomposition of 
process 50 is intended to calculate the matrices c^"^, d^"^ 
according to the following relationship: 



where .c " and d " are matrices of coefficients of the 
low- frequency and high-frequency components of the input 
signal, respectively. 

Attention is directed to Figures 10 and ii in 
combination for a detailed description of this technique. 
Figure 10 is a flow chart illustrating the matrix algebra 
procedure for performing processes 50 and 52 of Figure 7 
(the algebra being the same in each case) . Figure 11 is a 
chart illustrating the relationship among the decomposition 
matrices used in performing processes 50, 52 according to 
the flow chart of Figure 10. 

As shown in Figure 10, the first step of the 
decomposition begins with the process 60 which interpolates 
the function f, for each horizontal row, using a B- spline 
function 0(x), to provide a matrix representation {c}^ of 
the input image block 51 using the following relationship: 




[32] 



.N-1 



i=-in+l 



[33] 
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where m is the decree of the spline, where k is the number 
of sample points within the interval (i.e., the length of 
the row of image block 51, in pixels) , and where the B- 
spline function (p-^^ (x) is of the form: 

5 

2 

(i>i{x) =2 2<j)(2-?"^-;c) 

(34] 

According to the preferred embodiment of the invention, it 
10 has been observed that the accuracy of the decomposition 
and the stability of the result of odd degree splines 
(i.e., linear, ciabic, etc.) is improved relative to even 
degree splines (i.e., quadratic). As such, it is preferred 
that m, in equation [31] be an odd number (m = 1, 3, 5, . 
IS . .. ) . 

Upon determining the matrix c^, process 62 (Figure 10) 
is then next performed to compute the coefficients of a 
matrix {a}^, using the relationship a^ = C^c^. According 
20 to this preferred embodiment of the invention, the matrix 
is the inner product of the cardinal spline function 
<t> (X) : 

25 [35] 

The calculation of process 62 returns the matrix a^, as 
shown in Figure 11. 
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Procees 64 is then performed in order to compute 
intermediate decomposition coefficients utilizing the 
spline and boundary-wavelet concepts. These intermediate 
decomposition coefficients are also obtained by way of 
matrix algebra using digital matrix processor 30 of 
compresBor 16, shown in Figure 5. For the low- frequency, 
low-pass, or spline component, the intermediate 
decomposition coefficients a^"^ are calculated using the 
following matrix operation: 

136] 

where P^"^ is a known matrix that contains coefficients of 
the B-spline to be applied to the input image block, and 
which has the following form: 



[37] 



The elements p of the matrix of equation {37] are defined 
as follows: 

Pl,^ = i^li 

[38] 

for 1 from -m+l to -1, and for k from m+21 to -m+i, where 
represents the B-epline of order m, on the zeroth 
level and with ith initial knot. 
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For the high-frequency, band-pass, or wavelet 
component of the row of the image block 51, the 
intermediate decompoEition coefficients b'^"^ are calculated 
using the following matrix operation: 

[39] 
E the 

boundary- wavelets to be applied to the input image block, 
and having the form: 

140] 

where the elements q of the matrix are defined as follows: 

N _ i=i3 i=/r+l 
Qk, 1 — — 

[41] 

for 1 from -m+1 to -1, and for k from 3m-2+2l to -m+1, and 
where B®^ ^ represents the B-spline of order m on the 
2eroth level and with ith initial knot. In equation [41], 



ij = max (-ra+1, k-m) 

i- - min (2(l4-m-l), k+m-1) 
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3y way of explanation., the sum of the a terms in equation 
[41] corresponds to the boundary wavelets for those points 
in image block 51 that are near the edges, and the sum of 
the q terms in equation [41] corresponds to the inner 
wavelets for those points in image block 51 that are away 
from the edges. 

According to the preferred embodiment of this 
invention, each of the matrices C^, P^'^ and Q^"^ are pre- 
calculated and stored in program data memory 35 in each 
channel compression subsystem 29. in a manner accessible by 
digital matrix processor 30, so that the matrix operations 
62, 64 of Figure 10 can be readily and rapidly performed. 
The Appendix to this specification specifies examples of 
these matrices C^, P^'^ and Q^'^ which have actually been 
used for decomposition of an image block of a size 256 
pixels square, and where the numeric representation of 
matrices C, P and Q in the Appendix are given for the ciibic 
(m=4) case. 



Upon obtaining the intermediate decomposition 
coefficients in process 64, the final decomposition 
coefficient matrices c^"^ and d^"^, for the low-pass 
(spline) and band-pass (wavelet) components, respectively, 
are obtained by processes 66, 68, respectively. According 
to this preferred embodiment of the invention which 
utilizes matrix algebra, process 66 obtains the final low- 
pass decomposition coefficient matrix c^"^ from the 
relationship: 

^N-l ^ ^-l^N-l 

?? 

where the matrix C^"^ is the inner product of the cardinal 
spline function ^ (x) , calculated according to equation [35] 
noted above for the next matrix in sequence. The values of 
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frequency portion of processed image block 53 of Figure 9b, 
and thus represent the function fj^_^(x,y) in the spatial 
domain. Matrix c^"^ therefore also represents the 
frequency domain representation fj^-l shown in Figure 8. 

Similarly, process 68 of Figure 10 determines the 
final decomposition coefficient matrix d^"^ for the band- 
pass component of the image block 51. Process 68 is based 
upon the operation: 

b = D d [43] 

where matrix D^"^ is a known matrix that may be 
precalculated according to the relationship: 

« (qN)V*^(qN) [44] 



and is thus based upon known relationships. Upon 
20 completion of process 68, the matrix d^'^ corresponds to 
the high-frequency values in processed image block 53 of 
Figure 9b. The matrix d^~^ thus also represents the 
spatial function gj^.^^^^'Y)' its corresponding frequency 
domain representation g^^, which is the band-pass component 
25 shown in Figure 8. 



Upon completion of the horizontal decomposition of 
process 50, matrices c^~^ and d^"^ are stored in memory 
banks 36 of channel decomposition subsystem 29. preferably 

30 in the order illustrated in Figure 9b as processed image 
block 53. Referring back to Figure 7, process 52 is then 
performed to again decompose processed image block 53 in 
similar manner as process 50, only in a vertical direction, 
i.e., for each vertical column of processed image block 53, 

35 taken column-by-column. In other words, the decomposition 
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of process 52 is performed considering the image data for 
each vertical column of processed image block 53 as a one- 
dimensional spatial function in the y-dimension. According 
to the preferred embodiment of the invention, the matrix 
5 algebra used in process 52 is identical to that shown in 
Figures 10 and 11, for the next level of matrices in 
sequence, except on a column-by- column basis for each 
vertical column in processed image block 53. 

10 Process 52 thus decomposes the low- frequency spatial 

function fjj_^(x,y) into a low-frequency portion "LL" and a 
higher- frequency portion "LH" contained within processed 
image block 55 as shown in Figure 9c, where the second 
decomposition is performed on processed image block 53 in 

15 a vertical direction. Figure 8 illustrates, in the 
frequency domain, that the matrix portion LL corresponds to 
the function i^-2{f^-l) that is the low-pass portion of the 
prior low-pass function fjj-1, and that the matrix portion 
HL corresponds to the function ^-Ifj^-l that is the band- 

20 pass portion of the prior low-pass function fj^-l. 

Similarly, process 52 decomposes the prior band-pass 
decomposition function g^^-l into a low- frequency (i.e., 
lower band-pass) portion ("HL" in Figure 9c) and a high- 

25 frequency portion ("HH" in Figure 9c), but where the second 
decomposition is performed on processed image block 53 in 
a vertical direction. Referring to Figure 8, this 
decomposition provides frequency domain representation 
fj^-2gjg-l corresponding to the HL portion of final processed 

30 image block 55, and frequency domain representation gj^-2gjj-l 
corresponding to the HH portion of final processed image 
block 55. 

The LL portion of final processed image block 55 
35 corresponds to the . "blur" image of the input video image 



wo 95/1 96M 



PCTAJS9M)0563 



54 

block 51, and the LH, HL and HH portions of final processed 
image block 55 correspond to band-pass portions of the 
input video image block 51, taken in both the horizontal 
and vertical directions, successively. As is evident from 
the frequency domain representations of Figure 8, the LH, 
HL and HH components correspond to the components of the 
image in increasing frequency bands, with the HH component 
corresponding to the highest frequency component. 

The decomposition performed in processes 50, 52 
described hereinabove provide important benefits in the 
field of video image decomposition. Firstly, the benefits 
of time -frequency localization of the signal in a manner in 
which the frequency- width of the window widens and the 
time -width of the window shrinks with increasing frequency 
provided by wavelet decomposition are obtained. This 
allows for more detailed and thorough analysis of rapidly 
changing portions of the signal, and thus a more accurate 
decomposition of the video image. Secondly, by using the 
boundary-spline-wavelet approach described hereinabove, 
boundary effects at the edges of the image or of subdivided 
image blocks within the image are eliminated. The 
elimination of these boundary effects allows for smaller 
capacity computers to perform the decomposition with no 
degradation in image quality. Elimination of boundary 
effects also enables each subdivided image to be processed 
independently by a stand-alone device or processor system, 
so that the entire decomposition may be performed by 
parallel processing or parallel computing, without 
extensive modifications. In addition, the decomposed image 
may, upon decompression, be magnified without the presence 
of boundary effect artifacts in the displayed image. 
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Numerical decomposition examplp 

The decomposition of processes 50, 52 can be 
illustrated by way of a simple numerical example. In this 
5 example, the input image block ie eight -by-eight pixels in 
size, and contains an eight-bit digital value 
representative of the display intensity at that location 
(i.e., is in PGM format). An example of the input image 
block in this format is as follows: 

10 
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Process 50, as noted above, performs a boundary- 
spline -wavelet decomposition of the input image block, 
taken for each horizontal row. In this example, where a 
trivial zero degree (m=0) wavelet is used, for purposes of 
25 explanation, the horizontal decomposition of process 50 
results in the matrix: 
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The portion of the above matrix on the left-hand side of 
the divider corresponds to the low- frequency portion of the 
input image, while the portion of the above matrix on the 
right-hand side of the divider corresponds to the high- 
5 frequency portion of the input image, both performed in a 
horizontal manner. 

Process 52 then decomposes the result of process 50, 
considering the data in vertical columns as one -dimensional 
10 spatial sampled functions. Again using the same trivial 
zero-degree wavelet, the vertical decomposition of process 
52 provides the result: 
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The orientation of the above matrix uses the nomenclature 
25 of Figure 9c, as follows: 

LL j HL 

30 

LH j Jffl 

35 
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Video imaoe decomposition example 

Referring now to Figures 12a through 12e, an example 
of the boundary-spline-wavelet decomposition according, to 
5 the preferred embodiment of the invention as applied to an 
actual video image will be described. Figure 12a 
illustrates an input video image in PGM format to which 
boundary-spline-wavelet decomposition is applied. Figures 
12b through 12e illustrate the LL, LH, HL and HH components 

10 of the final processed video image after performing 
processes 50, 52 of Figure 7, using the matrix operations 
described hereinabove relative to Figures 10 and 11. As is 
evident from Figure 12b, the LL component is a quite 
faithful representation of the input image of Figure 12a. 

15 In the LH, HL, HH components illustrated in Figures 12c 
through 12e, respectively, the visible striations 
correspond to higher frequency components of the input 
image . 

20 

b. Thresholding and compression 

Referring back to Figure 6, upon the completion of the 
horizontal and vertical decomposition of the input image 
25 block performed in process 40, process 42 is next performed 
by quantization processor 32 upon the results of the 
decomposition stored in memory banks 36, 

Specifically, the LH, HL and HH components of the 
30 decomposition are subjected to thresholding and 
quantization in process 42, with no such quantization 
performed upon component LL. This selection of the LH, HL 
and HH components for quantization is based upon the 
observation that most real-world images in PGM format will 
35 consist of low- frequency intensities, and that the higher- 
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frequency wavelet, portions of the decompoeition will tend 
to have a large number of small or zero coefficients. The 
numerical example set forth above relative to the 
decomposition of the eight-by-eight image illustrates this 
effect. In addition, the exemplary image of Figures 12a 
through 12 e also shows that the LL component of the 
decomposition contains the most information, with the LH, 
HL and HH components containing only a small amount of 
significant information indicated by the striations 
therein. 

According to the preferred embodiment of the 
invention, therefore, this large population of small values 
in the higher- frequency components after decoir^osition may 
be either discarded (or reduced in memory requirements) by 
the quantization of process 42. For example, with 
reference to the above numerical example, all coefficients 
having an absolute value less than 2.0 may be set to zero 
(thresholding) , and all remaining coefficients may be 
rounded to their nearest signed integer value. The memory 
requirements for storage of the coefficients that undergo 
the thresholding and quantization are thus much reduced, 
even before the application of lossless compression 
techniques as will be noted below. 



Other thresholding and quantization techniques may 
alternatively be applied to the decomposed image data, 
selecteible according to the code stored in quantization 
data register 31 described hereinabove. 
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Video imaoe quantization example 

Referring now to Figures 13a through 13c, an 
illustration of the result of thresholding performed' in 
5 process 42 is illustrated, relative to the video image 
decompression . example of Figures 12a through 12e. Figure 
13a is the same image as the LL "blur" component also shown 
in Figure 12b. Figure 13b illustrates, for example, the 
sum of the LH, HL and HH coefficients after the 

10 decomposition of process 4 0 of Figure 7, and thus is the 
sura of the images of Figures 12c through 12e discussed 
above. Figure 13c illustrates the sum image of Figure 13b 
after the application of a threshold limit and quantization 
of process 42. As is evident from Figure 13c, the higher 

15 frecfuency components LH, HL, HH from the decomposition of 
process 4 0 correspond only to the edges and other sharp 
transition locations of the input image. 

20 c. Completion of video compression 

Referring back to Figure 6, after the thresholding and 
quantization of the LH, HL, HH components of the decomposed 
input image performed in process 42, decision 43 determines 
25 whether the desired compression ratio has yet been reached. 

As illustrated particularly by the video and numerical 
examples noted above, upon the completion of a single 

30 decomposition process 40, the LL component of the image may 
be adequate to accurately convey the input image. This 
single decomposition of the image will provide up to a 4:1 
compression ratio, depending upon the memory requirements 
for the quantized high frequency results. This maximum 

35 ratio considers that the wholesale discarding of the higher 
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frequency components will leave only the LL component 
residing in a matrix that is one -fourth the size of the 
input image block. Specifically, therefore, decision 43 
determines if the compression ratio achieved so far is 
5 adequate for the desired transmission or storage and, if 
not, passes control back to the decomposition process so 
that the LL component resulting from the prior 
decomposition may again be decomposed into four more 
components . 

10 

According to this embodiment of the invention, the 
determination of whether the desired compression ratio has 
been obtained may be done relative to a predetermined 
compression ratio. In such a case, timing circuit 37 will 
15 maintain a count of the number of passes through 
decomposition process 40, and will perform decision 43 by 
comparing the resulting compression ratio against a 
previously stored value. 

20 Alternatively, the compression ratio and thus decision 

4 3 may be determined in a dynamic manner, frame by frame, 
depending upon the accuracy with which the LL component is 
representative of the input frame image. Conceptually, 
such a determination will be a measure of the difference 

25 between the LL component and the input image block relative 
to a predetermined accuracy limit, such that if the LL 
decomposition is within a preselected c limit, an 
additional pass through the decomposition process 4 0 may be 
performed. It is contemplated that this determination can 

30 be made automatically by main controller 28 in decompressor 
16 so that the process performed in each of channel 
compression subsystems may remain consistent, by 
calculating a numerical value based upon the sum of the 
coefficients in the LH, HL, and HH components of the 

3 5 decomposed image, such sura indicating the difference 
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between the input image to that pass of the decomposition 
process and the resultant LL component. 

In the usual case, where each video image frame is 
subdivided into smaller image blocks for compression, it is 
preferred that the compression ratio be constant for all 
compressed image blocks of the frame. Accordingly, 
decision 43 is preferably performed after each image block 
has been decomposed and quantized in processes 40, 42, to 
allow the dynamic determination of whether to repeat the 
decomposition process 40 is made based on the worst case 
decomposed image. 

According to this dynamic determination of decision 
43, video image frames that do not contain high frequency 
information to any large extent, such as background images 
with smooth transitions between colors, may be compressed 
to a significantly higher ratio than can video image frames 
that do contain high-frequency information, or a large 
number of sharp transitions. Use of such dynamic 
compression can optimize the overall compression efficiency 
with mi^nimal impact on the image quality. 

Whether statically or dynamically determined, upon 
decision 43 returning the result that the desired 
compression ratio has been obtained, according to the 
preferred embodiment of the invention, lossless compression 
is then performed upon the results of the decomposed and 
quantized images for the frame, in process 44 of Figure 6. 
Referring back to Figure 5, the lossless compression of 
process 42 is preferably performed by lossless compressor 
34 according to a conventional lossless technique such as 
Huffman encoding. The lossless compression of process 42 
is especially beneficial in compressing the quantized 
higher frequency LH, HL, HH components from the 
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decomposition, considering that non-zero or varying values 
in these components will be quite sparse for most video 
image frames . 

After the lossless compression • of process 42, the 
compressed video image data is formatted, or coded, on a 
frame -by- frame basis for transmission or storage, as the 
case may be. As noted above, it is contemplated that most 
video image frames will be compressed according to the 
preferred embodiment of the invention after sijbdivision 
into multiple image blocks, such that a complete frame will 
consist of multiple ones of such blocks, transmitted 
together. As illustrated in Figure 4a, it is preferred 
that the coding of the compressed image data be performed 
within compressor 16, preferably by data flow interface 39 
shown in Figure 5, prior to its application to digital 
commvmi cat ions network 15. 

Referring now to Figure 14, a schematic illustration 
of formatted frame 70 of data according to the preferred 
embodiment of the invention will now be described. As will 
be described in further detail hereinbelow, the 
decompression of video image data compressed according to 
the present invention can occur very rapidly; for example, 
portions of motion pictures compressed according to the 
present invention have been decompressed and displayed on 
a real-time basis. This rapid and accurate decompression 
allows for the ability of enhanced features to be used in 
the display of the decompressed images. The construction 
of data frame 70 of Figure 14 includes the compressed bit 
stream data corresponding to a video frame, plus the 
appropriate formatting "header" information to allow the 
enhanced features to be operable. The header portion of 
frame 70 preferably provides complete information to 
describe the frame and its position. 



SBraMff<SKn5{BBl^))n 04/01/2005 



wo 9S/I96U 



PCT/US9MK»«3 



63 

As illustrated in Figure 14, frame 70 is a sequential 
block of data formatted for a storage device such as 
computer memory, disk storage, CD-ROM and the like. It is 
therefore contemplated that, in order to take advantage of 
all of the features of the present invention, compressed 
data frames 70 will be stored in a computer memory prior to 
its decompression, rather than decompressed in a real-time 
manner as received over digital network 15, Real-time 
decompression and display may alternatively be performed as 
the data is received, but certain ones of the features 
described hereinbelow will not be as useful in such a case. 
Since the extraction of the header information from frame 
70 requires extremely little computing time and effort, 
inclusion of this header information will have 
substantially no penalty in the overall performance of the 
real time decompression and display. 

The first portion of frame 70 is Hi header 71, which 
is data of string type used to specify the status of the 
bit stream compressed image, and as such may contain on the 
order of twenty bytes of data. For example, HI header 71 
may include an identifier of the movie of which the frame 
is a part, and may also contain a security code for 
preventing unauthorized viewing or use of the compressed 
data. For example, if the compressed video data 
corresponds to a movie being communicated over digital 
telephone lines, HI header may contain a code corresponding 
to an identifier of a local decompression system 20 so that 
only that decompression system will be enabled to 
decompress the data. Field 72 of frame 70 is a four-byte 
field of long integer type which contains the number of its 
corresponding frame 70 to specify the position of the frame 
20 in the entire video image sequence. Field 73 is a two- 
byte field of integer type which identifies the group of 
motion (if any) to which frame 70 belongs. A group of 
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frames 70 may be designated as a group, such that the 
frames in the group cannot be separated from one another or 
cut from the entire sequence. Such grouping can prevent 
unexpected side effects in the display of the motion 
5 picture sequence. 

Fields 74 and 75 then follow in frame 70 according to 
this example, to facilitate control of the display of the 
video sequence containing frame 70. Field 74 is a four- 

10 byte field of long integer type which contains the address 
at which the previous frame in the sequence begins, 
enabling rapid jumping back to the previous frame as 
desired. As will be described hereinbelow, a user control 
interface may be provided with decompressor system 20 to 

15 allow interactive control of the display of the video 
sequence, in which case field 74 will facilitate the 
skipping and selection of individual frames in reverse 
order. Similarly, field 75 is a four-byte field of long 
integer type which contains the address of the next frame 

20 in the sequence, allowing rapid skipping of frames 70 in 
the forward direction during decompression and display. 

Field 76 is a two-byte field of "integer type that 
indicates the complexity of the image contained within 
25 frame 70, by specification of compression ratio, quality 
index, or a user-defined specification of the image, such 
values useful in measuring and controlling the performance 
of the decompression and display. 

30 Fields 77, 79, 81, 83 indicate certain parameters used 

in the compression of the video data for frame 70. The use 
of header information to communicate these parameters allow 
for decompression system 20 to have selectable 
capabilities, such that it can decompress video image data 

35 that were compressed according to different techniques. In 
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addition, since each frame 70 includes these fields 77, 79, 
81, 83, the provision of these fields according to the 
preferred embodiment of the invention enables dynamic 
compression, such that different compression techniques may 
be used on different frames 70 in the same video sequence. 
Field 77 is a two-byte field of integer type that specifies 
the level of decomposition in the compressed data of frame 
70, so that decompression system 20 may perform the proper 
levels of decompression upon receipt. Field 79 is an 
integer field indicating the type of lossless compression 
performed by lossless compressor 34, for example: 

no lossless compression 
Huffman coding 
adaptive Huffman coding 

and so on. Field 81 is a two-byte field of integer type 
that indicates the quantization mode used by quantization 
processor 32 described hereinabove relative to compression 
system 10, for example following the tabular listing of the 
codes noted above. Field 83 is a two-byte field of integer 
type that stores the quantization value used by 
quantization processor 32, and thus depends upon the mode 
indicated in field 81. Using the tabular example noted 
hereinabove for control of quantization processor 32, the 
value of field 83 will indicate the quantization value as 
follows : 
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Mode M (field 81) Field 63 represents: 

0 0 (don't care) 

1 thresholding value 

2 dequantizing scalar 

3 address of the JPEG 

quantization table, 
as an offset from 
the compressed data 

4 address of vector 

quantization table, 
as an offset from 
the compressed data 

As indicated in Figure 14, portion 78 of frame 70 
contains the compressed image data for field 70, and 
follows fields 87, 89. Field 87 is a four-byte field of 
long integer type that specifies the length of portion 78 
containing the compressed data stream, and is useful in 
assisting the reading of the compressed data stream, such 
as by way of a DMA operation. Field 89 is a four-byte 
field of long integer type that indicates the offset 
distance between field 87 and the start of portion 78 
containing the compressed image data; field 89 may thus be 
used to reserve space within frame 70 for other fields and 
information defined by the user or other developers. 

Portion 78 contains a data stream of the compressed 
video image data for frame 70, as noted above. The length 
of portion 78 will, of course, depend upon the compression 
ratio obtained in the compression process, upon the number 
of pixels in the frame, and also upon the extent to which 
the lossless compression of process 44 compressed the 
higher frequency components of tlie frame. In addition, it 
is contemplated that audio information may also be encoded 
in a conventional manner, and placed within portion 78 of 
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frame 70, to support the transmission or storage of sound 
motion pictures. 

This arrangement of frame 70 is particularly useful, in 
the interactive decompression and display of a sequence of 
video frames. Specifically, fields 74, 75 and 76 enable 
decompressor 20 to flexibly display the frames in the 
sequence, especially in the case where the sequence of 
frames 70 are sequential frames in a motion picture. For 
example, decompressor 20 can interrogate field 76 to 
determine if the processing capacity of decompressor 20 and 
its display system 26 is such that every frame in the 
sequence cannot be decompressed and displayed in real time; 
if so, decompressor 20 can skip to the next frame 70 in the 
sequence indicated by the contents of field 75 in frame 70. 
While the quality of the displayed motion picture will be 
reduced from the best possible images when frames are 
skipped, those frames that are not skipped are displayed in 
real-time, so that the time -dependence of the motion in the 
motion picture is accurately conveyed. 

Fields 74, 75, 76 also provide interactive display 
capability. As field 74 indicates the address of the 
previous frame in the sequence, frame 70 allows the 
capability of backwards display of a motion picture, 
whether for each frame 70 in the sequence or with frames 
skipped as described above based on the time required for 
decompression and the capability of decompressor 18 . in 
addition, the information provided by field 76 facilitates 
the synchronization of the display of the sequence of 
frames 70, and also allows for easy scaling of the time 
base to provide slow-motion or enhanced -speed display. 

Accordingly, referring back to Figure 6, the 
formatting or coding of process 4 6 thus prepares the 
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compressed video image data for transmission over digital 
network 15 for storage, decompression and display by 
decompression system 20, or alternatively to disk unit 22 
for archival storage. 

5 

4 ■ Construction of the decomtsressor 

3.= Hioh performance decompressor 

10 

Referring now to Figure 15a, the construction of 
decompressor 18 according to the preferred embodiment of 
the invention will now be described in detail. It is, of 
course, contemplated that other architectural arrangements 

15 of circuitry may be used to decompress video data according 
to the present invention. Specifically, it is contemplated 
that a conventional general purpose computer system may be 
capable of performing the compression of the video image 
data according to . the present invention. However, the 

20 example of Figure 15a incorporates a preferred embodiment 
of the circuitry for performing the data decompression 
functions described herein. 

In this example, decompressor 18 preferably includes 
25 a data flow interface 80 coupled to network 15 to receive 
the incoming bitstreara data of the compressed video image, 
as described above relative to Figure 14 . Data flow 
interface 80 provides an interface to network 15, and 
serves to separate the R, G, and B components of the 
30 incoming bitstream into three channels, namely the R- 
stream, G- stream and B- stream channels. Decompressor 18 
also includes main controller 84 and three siibstantially 
identical channel decompressor subsystems 88j^, 88^, 88g. 
Main controller 84 preferably controls the operation of the 
35 functional circuitry within decompressor 18, and as such is 
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connected by way of the appropriate bidirectional buses and 
control lines to subBystems 88 and data flow interface 80 
to control the timing, feedback and receiving operation of 
decompressor 18, Main controller 84 is preferably a 
general purpose programmable microprocessor or other 
central processing unit (CPU) of sufficient computational 
power and capacity to process some or all of the image data 
and to control the performing of the image compression 
functions to be described hereinbelow. It is contemplated 
that microprocessors having performance levels similar to 
or greater than those of the 80486 type (including those 
available from Intel Corporation or Cyrix Corporation) , of 
the 68040 type (including those available from Motorola) , 
and of the SPARC processor type (available from Texas 
Instruments Incorporated or Sun Microsystems, Inc.) will be 
suitable for use as main controller 84 in decon^jressor 18. 

The three channel compressor siibsystems 88j^, 88^, 88^ 
each receive a corresponding one of the separated channels 
presented by data flow interface 80, so that the boundary- 
spline-wavelet decompression of the three color . components 
is performed in parallel according to this embodiment of 
the invention. The construction of chaiuiel decompressor 
subsystems 88 is substantially the reverse of that of 
channel compressor subsystems 29 described hereinabove. 
Each of subsystems 8 8j^, 88^, 88g is therefore constructed 
substantially identical to perform these similar tasks as 
the others, in parallel; for purposes of clarity, the 
following description is provided only for channel 
decompressor subsystem BBj^, it being understood that the 
construction and operation of the other subsystems 88 will 
be similar, if not identical, to subsystem 88j^ described 
herein. 
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Each channel decompressor subsystem 88 according to 
this embodiment of the invention is specifically designed 
to perform the functions of lossless decompression, 
dequantization, and boundary spline wavelet re -compos it ion 
5 (or decompression) , ae used in the decompression operation 
according to this embodiment of the invention. Channel 
decompressor subsystem 88 according to this embodiment of 
the invention includes the circuit functions of digital 
matrix processor 86, timing circuit 107, dequantization 
10 processor 108, lossless decompressor 82, program data 
memory 112, and multiple memory banks 110 for storage of 
image data . 

The incoming data received by subsystera 88, as shown 

15 in Figure 15a, is received by lossless decompressor 82. 

Lossless compressor 82 may be implemented by way of a 
conventional digital signal processor such as the TMSC4 0, 
programmed in such a manner as to perform lossless 
decompression upon the incoming data stream for that 

20 channel as presented by data flow interface 80. The 
lossless decompression performed by lossless decompressor 
82 may be according to a conventional technique, such as 
Huffman encoding, adaptive Huffman encoding, arithmetic 
encoding, LSQ coding, and the like; specifically, however, 

25 lossless decompressor 80 must either recognize (from field 
79 of frame 70) or be previously programmed to operate 
according to the lossless compression technique used in 
compression of the incoming data. Lossless decompressor 82 
stores the decompressed incoming image data in memory banks 

3 0 110, in preparation for the re- composition of the image 
data. Alternatively, lossless decompressor 82 may be 
implemented as a custom logic circuit for providing this 
function. 
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The output from- lossless decompressor 82 corresponds 
to the decomposed image data for that channel (R, G, B) . in 
identical form as that presented to lossless compressor 34 
in compressor 16, as described hereinabove. Dequantizat ion 
5 processor 108 is a logic circuit or programmable device 
such as a microprocessor, for de-qfuantize the output from 
lossless decompressor 82. According to this preferred 
embodiment of the invention, dequantizat ion processor 108 
is controllable to operate according to various 

10 quantization or thresholding modes, according to the 
contents of dequantization code register 109 which receives 
the contents of field 81 of frame 70. The dequantization 
value in field 83 is used in the dequantization perfonned 
by dequantization processor 108. The results of the 

15 dequantization are stored in memory banks 110. 

Digital matrix proceesor 8€ performs the boxawiaxy- 
spline-wavelet composition upon the results of the 
dequantization, using pre-calculated matrices stored in 

20 program data 112. The procedure for this composition is 
described in detail hereinbelow. The results of the 
operations performed by digital matrix processor 8£ are 
stored in memory banks 110. As in the case of compressor 
16, digital matrix processor 86 may be implemented by way 

25 of a conventional digital signal processor, such as the 
i860 microprocessor available from Intel Corporation and 
the TMSC40 digital signal processor available from Texas 
Instruments Incorporated, or by way of a general purpose 
microprocessor such as those of the 80386 and 80486 

30 families available from Intel Corporation, and of the 6 8030 
and 68040 families available from Motorola. 

Timing circuit 107 performs the functions of 
controlling the storage of the channel data from lossless 
35 decompressor 82, and the siibsequent retrieval thereof for 
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dequantizaticn and decompression processes. Upon 
completion of the decompression of the channel data for a 
frame, timing circuit 107 is operable to present the 
decompressed PGM channel data to main controller 84, for 
formatting and transmission to format converter 24 for 
display at display 26. 

Main controller 84 receives the decompressed channel 
data from each of subsystems 88j^, 88^, 88g and sequences 
the data into the format suitable for receipt by format 
converter 24. In addition, main controller 84 operates to 
provide feedback to data flow interface 80, such that upon 
decompression of a frame of data, data flow interface 80 
can be controlled to comm\inicate the next frame of data to 
subsystems 88 for decompression. 

In addition, main controller 84 is coupled to user 
interface device 85, which may be a mouse, trackball, pen 
input, keyboard or the like; in addition, user interface 
device 85 may be implemented as another computer, such as 
a personal computer workstation, through which a user 
controls the decompression and display of video data. As 
noted hereinabove, the construction of frame 70 shown in 
Figure 14 facilitates the interactive control of the 
decompression process, such that the user may control the 
direction (forward/backward) and rate of decompression and 
display of the sequence of video, image frames, using user 
interface device 85. Main controller 84 is thus operable 
to communicate to data flow interface 80 the order and 
selection of frames for decompression; it is therefore 
useful to incorporate adequate memory for the storage of 
multiple frames either within or accessible by data flow 
interface BO to allow such interactive decompreseion and 
display. 
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This example of decompressor 18 is intended to support 
the compression of high definition real-time true color 
video image data, where "true color" indicates the use of 
twenty- four bits of color information for each pixel, 
5 resulting in 16 17 million possible colors. The 
decompression rate for decompressor 18 is intended to be on 
the order of thirty frames per second, so as to support 
"real-time" video image decompression and display. As 
noted above, if the color and frame rate requirements are 

10 reduced from real-time true color video, it may be possible 
to implement decompressor 18 as a single channel, i.e. with 
a single channel compression subsystem 88. In this 
implementation, color data could be decotnpressed 
sequentially for the R, G and B components \inder the 

15 control of main controller 84 . In addition, if the frame 
rate permits, digital matrix processor 86 may be used to 
perform the lossless decompression, as well. 

The construction of decompressor 18 described 
20 hereinabove is intended as a "high-end" product utilizing, 
in a parallel fashion, three processing' boards for the 
three R, G, B channels. This design is preferably used in 
association with modem high-performance workstations such 
as the 4D series workstations available from Silicon 
25 Graphics, and SPARC workstations available from Sun 
Microsystems, as these workstations have performance levels 
adequate for delivering more than thirty frames per second 
of high resolution {800 pixels by 600 pixels) image data 
with 24 -bit true color capability. 
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b. Personal -computer expansion card decompression 

Referring now to Figure 19, decompression Bystetn 200 
5 according to an alternative embodiment of the invention 
will now be described in detail. Decompression system 200 
is intended for implementation on a conventional PC-based 
workstation, and is configured as a single 8-bit expansion 
board; alternatively, system 200 may be configured as a 16- 
10 bit ISA or EISA expansion board, or as a MCA (Micro Channel 
Architecture) board. In each case, it is contemplated that 
decompression system 200 will be suitable for medium 
resolution displays. 

15 Decompression system 200 includes lossless 

decompressor 202, which is in^lemented as a video 
processing board. Lossless decompressor 202 has an input 
port coupled to network 15 (or, of course, to disk storage 
or another source of video image data compressed according 

20 to the method described hereinabove) . Lossless 
decompressor 202 includes digital signal processor. (DSP) 
204, which is a conventional digital signal processor such 
as the TMSC25 and TMSC30 types available from Texas 
Instruments Incorporated, or a general purpose 

25 microprocessor such as the 68020 or 68030 type availeible 
from Motorola. Program memory 206 is preferably EPROM or 
other conventional firmware storage, for storing the 
programs utilized by DSP 204 in performing the lossless 
decompression upon the received data. 

30 

Lossless decompressor 202 receives the compressed RGB- 
PGM data from network 15 and performs the necessary 
lossless decompression according to the lossless 
compression technique used (if any) in compressor 16. It 
3 5 is preferred that lossless decompressor 202 be capable of 
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perfonning decompression of various types, selectable by 
way of field 79 in frame 70, described hereinabove relative 
to Figure 14. Lossless decompressor .202 is also coupled to 
user interface device 205, to allow a human user the 
5 capability of controlling motion control in the 
decompression and display of a sequence of video images. 
Lossless decompressor 202 presents its output to 
decompression processor 210 for the boundary- spline -wavelet 
decompression. 

10 

Decompression processor 210 according to this 
embodiment of the invention includes processing capability 
to perform boundary-spline-wavelet decomposition according 
to the preferred embodiment of the invention described 

15 herein. The main controller of decompression processor 210 
is digital signal processor (DSP) 225, which is preferably 
a high-performance single-chip digital signal processor, 
such as the TMSC40 available from Texas Instruments 
Incorporated or the i860 processor available from Intel 

20 Corporation, or a high-perf ormajice general purpose 
microprocessor such as the 80486 microprocessor available 
from Intel Corporation. Decompression processor 210 also 
includes program data memory 212 for storage of program 
code and pre -calculated matrices useful in the 

25 decompression process, data memory banks 214 for storage of 
image data, and digital matrix processor 220 for performing 
matrix operations useful in the decompression routine. 
Digital matrix processor 220 is preferably a conventional 
digital signal processor, such as the i860 microprocessor 

30 available from Intel Corporation and the TMSC40 digital 
signal processor available from Texas Instruments 
Incorporated, or a general purpose microprocessor such as 
those of the 80386 and 80486 families available from Intel 
Corporation, and of the 68030 and 68040 families available 

35 from Motorola. Host interface controller 208 is also 
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provided within decompression processor 210, for 
coTnmuni eating with a host computer (not shown) including 
the video display associated therewith. Program memory 222 
is also provided in decompression processor 210 and 
5 contains the program code for boundary spline wavelet 
reconstruction and format conversion. Video timing control 
circuit 217 is also contained within decompression 
processor 210, for controlling the viewing of the 
decompressed data as well as the timing of the 
10 decompression, so that a sequence of video frames may be 
viewed in real-time as a movie, or under the control of 
user interface device 205. 

In operation, lossless decompressor 202 stores the 
15 results of the lossless decompression for a frame in memory 
banks 214. These data are retrieved by DSP 225 via memory 
interface 224, according to a sequence stored in program 
meraory 222, and forwarded to digital matrix processor 220 
along with precalculated matrices stored in program data 

2 0 212 along the coefficient bus shown in Figure 19. Digital 

matrix processor 220 performs matrix operations upon the 
image data presented by DSP 225 to perform the boundary- 
spline -wavelet decomposition and forwards the results to 
DSP 225 along the data-out bus shown in Figure 19 for 
25 storage in memory banks 214. Digital matrix processor 220 
also provides control information to DSP 225 on the 
feedback bus therebetween. 

Upon completion of the boundary- spline -wavelet 

3 0 reconstruction or decompression, the decompressed image 

data for a frame is stored in memory banks 214 . Memory 
interface 224 enables the interleaved or simultaneous 
accessing of memory banks 214 for boundary- spline -wavelet 
decompression and display; host interface controller 208 
35 accesses memory banks 214 for display purposes. Host 
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interface controller 208 may also have sufficient 
capability to directly display the contents of memory banks 
214 containing the decompressed video image data; 
alternatively, the decompressed image data may be forwarded 
5 by host interface controller 208 to memory of the host PC 
for eventual display, by way of DMA. 

It is contemplated, of course, that still further 
alternatives to the decompression hardware may be utilized 
10 according to the present invention. 

5^ EOUndarV-spling-wavelet video imaoe daf a decomn-rpss-ign 

and display 

15 

Referring now to Figure 16, a method according to the 
preferred embodiment of the invention, for decompressing 
video image data that was compressed by way of boundary- 
spline-wavelet compression as discussed above, will now be 

20 described in detail. This decompression will be described 
relative to the hardware decompression system of Figures 4a 
and 15a, it being understood that analogous operations 
would be carried out by decompression system 200 
illustrated in Figure 19, for its particular 

25 implementation. 

The initial step of the decompression method of this 
example is the reading of the compressed file or sequence 
of frames 70 by data flow interface 80, indicated in Figure 

30 16 by process 90. Process 90 may be the result of the 
receipt of communicated compressed video image data by data 
flow interface 80 directly from digital network 15. 
Alternatively, where decompressor 18 is a portion of a 
stand-alone computer, disk unit 22 or another fixed storage 

35 unit may contain the file of compressed video image frames 
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from an earlier transmission, or as archival storage. In 
either case, the result of process 90 is the receipt of a 
sequence of compressed video image frames 70 by data flow 
interface 80, from which the remainder of the process of 
£ Figure 16 can retrieve individual frames 70. 

Process 91 is next performed, where each of the 
channels of a selected frame 70^ undergoes lossless 
decompression by lossless decompressor 82. As noted above 

10 relative to Figure 15a, the channels of data are 
deccmpressed in parallel according . to the preferred 
embodiment of the invention, and as such the process flow 
of Figure 16 from this point forward will be described for 
a single channel (R, G, B) of frame 70^^. The particular 

15 frame 70^ is selected according to the sequence of frames 
70 in the motion picture, or is alternatively selected by 
the user via user interface device 85. In either case, 
main controller 84 controls the access of selected frame 
70^, so that the appropriate frame 70 is presented to 

20 channel decompressor svibsystems 88, which provide the data 
for frame 70^ to lossless decompressor 82. As noted above, 
the losslesB decompression performed in process 91 by 
lossless decompressor 82 corresponds to the type of 
lossless compression performed by lossless compressor 34 in 

25 compressor 16. The result of the lossless decompression of 
process 91 thus provides the final decomposed image blocks 
55 of frame 70^, quantized as described hereinabove 
relative to the compression method. 



30 After the lossless decompression of process 91, the 

image data is dequantized for each channel R, G, B by 
deguantization processor 108 in its corresponding channel 
decompression subsystem 88. The type of dequantization 
performed by dequantization processor 108 is controlled by 

35 the contents of dequantization data register 109, which 
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contains a value ccrrespcnding to that transmitted as field 
81 in frame 70^. In the event that the data was quanti2ed, 
the guanti2ation value transmitted in field 83 of frame 70 

n 

is utilized to restore the data to that prior to 
5 q-jantization in compressor 16. 

Upon completion of the deguantization of process 92, 
process 93 is performed for each image block of frame 70^ 
to reconstruct the full video image in PGM format, with 

10 each pixel location containing a value corresponding to the 
intensity at that location. In effect, process 93 reverses 
the decomposition process described hereinabove relative to 
Figures 6 and 7, according to the boundary -spline -wave let 
approach. This reconstruction of process 93 is performed 

15 on a block'-by-block basis, for each of the image blocks in 
the selected frame 70^. Upon completion of process 93, the 
selected frame 70^ is stored as a PGM format frame in 
memory banks 110. 

20 Referring now to Figure 17, a method for 

reconstructing the selected frame 70^ according to the 
preferred embodiment of the invention will now be described 
in further detail. Recalling that the decomposition of the 
video image data performed during compression is performed 

25 first in the horizontal direction and then in the vertical 
direction, the decomposition must be performed in the 
reverse order. The first step in the reconstruction of the 
video image data is therefore the reconstruction of each 
column in the vertical direction, as indicated by process 

30 98. 

The reconstruction process 98 is also most 
efficiently, and thus preferably, performed by digital 
matrix processor 86 in the corresponding channel 
35 decompression subsystem 88 using matrix algebra operations 
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similarly as in the compression process described 
hereinabove. Accordingly, process 98 begins by considering 
the image block in columns, each containing a set of 
N-k 

coefficients c for the low frequency portion and a set 
5 of coefficients d^"^ for the high frequency portion, with 
k being the number of times that boundary- spline-wavelet 
decomposition was performed during compression to obtain 
the desired compression ratio. For example, where the 
compression ratio was 4:1, the value of k would equal 2 
10 (i.e., both a row and a column decomposition was performed 

once); for a compression ratio of 64:1, k would equal 6. 

Figure 18 illustrates the relationship of the matrices 
used in the reconstruction of the image block performed in 
15 process 98. Each reconstruction operation (process 98) is 
directed to obtaining the next higher order coefficient 
values Cj,_, - 

Conversely to the case of compression, the 
matrix operation is equivalent to: 

20 C^-^*^ = pN-k^N-k ^ QN-k^N-k 

where the matrices and are the spline and wavelet 
matrices, respectively, for level N, as used in the 
deccmposition and compression process. As noted above, 
25 matrices and Q*^ can be pre-calculated and stored in the 
program data memory 112 of channel decompression subsystems 
88. 

Process 98 thus performs one level of the matrix 

30 operations of equation [45] on a column-by- column basis for 

each column in the image block. Upon completion of process 

98, process 100 performs a similar reconstruction as that 

in process 98, only on a row-by- row basis for each row in 

the image block. As such, the operation of equation [45] 

N-k+2 

35 is performed to obtain the coefficients c 
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It will, of course, be noted that the decompression of 
the video image data according to the preferred embodiment 
of the invention will not result in an exact duplicate of 
the input image presented to the compression process, as 
5 the effects of the quantization and thresholding process in 
the ccmpression of the video image data cannot be full 
recovered (i.e., some data is lost). As such, the 
boundary-spline-wavelet compression according to the 
present invention is a lossy compression technique. It is 
10 contemplated, however, that the use of the boundary- spline- 
wavelet approach described hereinabove causes the loss of 
only that data at high-frequencies, such that the loss of 
quality of the image is minimized for a given compression 
ratio. 

15 

However, it should especially be noted that the 
decompression processes 98, 100 can be done quite rapidly, 
particularly with specialized circuitry such as a digital 
matrix processor performing the matrix operations noted 

20 above. This speed in processing arises from the wavelet 
forms selected in the compression, which correspond to 
express mathematical formulae and which are also quite 
adaptable to matrix operation techniques. As such, it is 
contemplated that the decompression performed by 

25 deccmpressor 18 can occur quite rapidly and, for many 
applications, need not require high performance 
microprocessor circuitry. 

Upon the reconstruction in both the column and row 
30 directions as performed in processes 98, 100, decision 101 
is performed to determine if the image block has been fully 
reconstructed (i.e., if the image block corresponds to a 
PGM format image, and does not contain low and high 
frequency components) . If not, the matrix operations of 



wo 9i/19683 



PCTmS9MX)563 



62 

processes 98, 100 are repeated to perform the next levels 
of reconstruction of the image for that image block. 

Upon process 100 completing the reconstruction of the 
5 image block, decision 103 tests whether more image blocks 
remain to be reconstructed in the selected frame 70^. If 
so, the next image block is retrieved from memory 80 
(process 104) and is reconstructed by processes 98, 100, 
101 as before. Upon completion of the reconstruction of 

10 all image blocks in selected frame 70^, process 106 is 
performed so that the reconstructed image blocks in frame 
70^ are stored in image buffer 88, arranged in the proper 
order to correspond to a full PGM format image frame, 
completing process 93 of Figure 16 for a single frame 70^^. 

15 The decompressed image data for the frame 70^, including 
that for all three channels R, G, B, are the forwarded to 
main controller 84 for communication to format converter 
24 . Main controller 84 then indicates completion of a 
frame to data flow interface and performs decision 95 to 

20 determine if additional frames are to be decompressed and 
display; if so, main controller 84 will select the next 
frame for decomposition in the sequence, or as directed by 
user interface device 85. 

25 Referring back to Figure 16, process 94 is then 

performed by way of which format converter 24 reformats and 
communicates the frame to display 26, in the desired 
fashion. It is contemplated that format converter 24 may 
perform such operations as gathering three successive PGM 
30 format frames, corresponding to the red, green, and blue 
color components of the same image, and presenting the data 
in RGB-PGM format to display 26; format converter 24 may 
also convert the RGB-PGM format data into another format, 
such as PCX, IMG, GIF, TIF, RLE, NTSC and the like, as 
■ 35 appropriate for display 26. 
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Referring now to Figure 15b, the construction of 
fcrmat converter 24 according to the preferred embodiment 
of the invention will now be described in detail. Format 
converter 24 includes three buffers 120j^, ^^0^, 120g for 
5 receiving from decompressor 18 and storing PGM format data 
corresponding to the three decompressed R, G, B channels. 
The outputs of buffers 12 0 are received by color combiner 
circuit 122, which is a data processing circuit such as a 
microprocessor, which combines the decompressed video data 

10 for the three R, G, B channels into RGB-PGM format. Color 
combiner 122 presents the results of this combination to 
the appropriate formatting circuitry for driving display 
26. In the analog case, color combiner 122 presents its 
output to video digital-to-analog converter (DAC) 126; 

15 video DAC 126 is a conventional video DAC suitable for 
receiving digital video or graphics inf ormation and driving 
analog display 26a. In the digital case, color combiner 
122 presents its output to color reducer and format encoder 
124 . Reducer/encoder 124 is a conventional circuit for 

20 encoding the RGB-PGM format data presented thereto by color 
combiner 122 into the desired format for digital display 
26d, such formats including TIF, IMG, GIF and the like. Of 
course, if display 26 is capable of directly displaying 
RGB-PGM video data, reducer/encoder 124 is unnecessary. 

25 

In addition, format converter 24 may also include 
graphics processing capability so as to perform more 
graphics processing operations, including magnification, 
zooming and the like. As noted above, the elimination of 

30 boundary effects resulting from the boundary-spline-wavelet 
compression and decompression according to the present 
invention is especially beneficial when such complex 
graphics processing features. For example, a portion of 
the image of frame 70^ may be readily magnified according 

35 to conventional graphics processing techniques , . such as the 
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use of interpolation techniques to fill in pixels between 
those for which the frame contains actual data, with a high 
degree of accuracy. In contrast, boundary effects that 
result from prior compression techniques tend to be 
exaggerated when the image is magnified, considering not 
only the magnified display but also the exaggeration 
resulting from interpolation between "true" image data and 
the boundary effect artifacts. 

Indeed, it is contemplated that the present invention 
will be especially beneficial for the transmission and 
storage of compressed video images for display on advanced 
displays that have higher pixel densities than the density 
of the image so compressed, due to the elimination of 
boundary effect artifacts by the present invention. In 
addition, it is contemplated that such magnification will 
allow for the display of the images on a plurality of video 
displays arranged in an array, with each display showing 
only a portion of the full image. 

Referring back to Figure 16, upon transmission of the 
selected frame 70^ to format converter 24 performed in 
process 94, decision 95 is then performed to determine if 
additional frames in the sequence are to be decompressed 
for display. If so, process 96 selects the next frame 70 
for decompression and display. According to the preferred 
embodiment of the invention, the selection of process 96 
may be made in multiple ways. A first approach for 
selecting the next frame for decompression and display is 
merely to select the next frame in the sequence, and access 
the same by way of field 75 (Figure 14) from the previously 
decompressed frame 70^; this approach will be followed 
during the decompression and display of a motion picture. 
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As noted above relative to Figure 14, field 76 
includes data indicating such factors as the compression 
ratio and the like that indicate the computational 
complexity required for the decompression and display of 
its frame. This allows for intelligent selection of the 
next frame in process 96, where the capability of 
decompression system 20 is limited relative to the amount 
of compressed video data in the transmission. For example, 
if the time that decompressor 18 will require for the 
decompression and .display and display of a frame is longer 
than the reciprocal of the frequency at which frames are to 
be displayed (e.g., longer than 1/30 sec. in the case of a 
conventional motion picture), main controller 84 will skip 
the next frame or frames, and select a later frame for 
decompression and display so that the sequence of frames 
that are displayed will appear in real-time fashion on 
display 26 . 

A third approach for the selection of the next frame 
in process 96 is that which may be directed by the user via 
user interface device 85. For example, the viewer may wish 
to display the sequence on a non-real-time basis to allow 
for intense study of each frame, in either a forward or 
backward direction, or the viewer may wish to repetitively 
view a selected portion of the sequence. These commands 
are conveyed to main controller 84 by user interface device 
85, and are used in process 96 according to the present 
invention to select the next frame. 

Upon the decompression emd display of all of the 
frames in the sequence as determined by decision 95, the 
process of decompression and display ends. 
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5. Conclusion 

The methods and systems for compressing and 
decompressing video image data described hereinabove 
5 relative to the present invention provide important 
advantages, as noted throughout the foregoing 
specification. These advantages include the fundamental 
benefits of wavelet analysis, where time -frequency 
localization of the input signal is implemented so that the 
10 time window narrows with increasing frequency and widens 
with decreasing frequency, thus providing highly accxirate 
analysis for transient periods of the signal, and avoiding 
the undersampling problems of conventional Fourier analysis 
techniques, 

15 

In addition, the video image compression and 
decompression techniques according to the preferred 
embodiments of the invention utilize boundary -spline - 
wavelets to eliminate boundary effects in the con^ressed 

20 images, specifically by using different wavelets for 
samples near the boundaries of an interval from those 
wavelets used for inner samples. The boundary wavelets do 
not require support from outside of the inter-zal; when 
applied to video compression according to the present 

25 invention, therefore, botindary effect artifacts are 
eliminated. This not only provides an extremely accurate 
representation of the original input image but, when 
coupled with the computational geometrical superiority of 
spline fimctions, also enables the performance of complex 

30 processing operations and display such as magnification, 
dynamic compression on a frame -by- frame basis, interactive 
display of a motion image, including insertion, editing and 
repetitive display. In addition, the present invention 
also enables slow display systems to skip frames so that a 
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motion picture can be displayed on a real-time basis with 
fewer frames per second. 

The elimination of boundary effects also enables the 
compression routine to be readily applied to subdivided 
blocks of the input image, without sacrificing image 
quality upon decompression and display. This allows for 
parallel processing techniques to be readily applied to the 
compression and decompression operations. In addition, 
lower-capacity computing equipment is enabled by the 
present invention to perform compression and decompression, 
while still obtaining high compression ratios and excellent 
picture quality. 

In addition, the present invention also enables a 
relatively higher compression ratio than prior systems, for 
a given accuracy, due to the improved time- frequency 
localization provided by the selected wavelets according to 
the present invention, and also the ability to compress to 
a desired accuracy limit as described above. The present 
invention further enables the dynamic compression of a 
sequence of frames, so that high frequency frames may be 
compressed to different ratios than low frequency frames. 

The present invention may be applied to various 
applications of video image compression, such as the 
transTOiseion of motion pictures and other video data 
between remote locations, such as over digital 
communication networks or by satellite transmission. The 
present invention may also be applied advantageously to the 
archival storage of video image data, both motion picture 
and still image data, in a compressed form. The archival 
storage of both motion video and still images is especially 
facilitated by the present invention due to the lack of 
boundary effects, as the decompression and display of the 
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compressed information may be performed in much higher 
density and capability systems than those used in the 
original compression, without exaggeration of artifacts 
such as boundary effects. 

5 

While the invention has been described herein relative 
to its preferred embodiments, it is of course contemplated 
that modifications of, and alternatives to, these 
embodiments, such modifications and alternatives obtaining 

10 the advantages and benefits of this invention, will be 
apparent to those of ordinary skill in the art having 
reference to this specification and its drawings. It is 
contemplated that such modifications and alternatives are 
within the scope of this invention as subsequently claimed 

15 herein. 
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WE CLAIM: 

1. A method of compressing digital data 

representative of a video inage for cciafflunication thereof, 
5 comprising the steps of: 

formatting the digital data corresponding to a 
frame of the video image into an array of locations in rows 
and columns, with the value at each array location 
corresponding to the intensity of the display; 
10 decomposing, in a first direction, each row of 

said array into low-frequency and high-frequency portions 
using boundary-spline-wavelets, and arranging the results 
of the decomposing into corresponding rows; 

then decomposing each column of said arranged 
15 results of the first decomposing step in a second 
direction, using boundary-spline-wavelets; and 

communicating the results of the decomposing 
steps to a receiving unit. 

20 2. The method of claim 1, wherein said communicating 

step comprises: 

storing the results of the decomposing steps in 
a fixed memory. 

25 3. The method of claim 1, wherein said commiinicating 

step comprises: 

transmitting the results of the decomposing steps 
to a decompression system for display. 

30 4. The method of claim 3, further comprising: 

after receiving the transmitted results of the 
decomposing steps, reconstructing the transmitted video 
image in the second direction, using boundary-spline- 
wavelets; 
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then reconstructing the results of the first 
reconstructing step in the first direction, using boundary- 
spline-wavelets; and 

displaying the results of the reconstructing 
5 steps on a video display. 



5. The method of claim 1, wherein the results of the 
second decomposing step comprise: 

an LL component, corresponding to the low- 
10 frequency component, taken in the second direction, of the 
low-frequency decomposition in the first direction of the 
first decomposing step; 

an LH component, corresponding to the high- 
frequency component, taken in the second direction, of the 
15 low-frequency decomposition in the first direction of the 
first decomposing step; 

an HL component, corresponding to the low- 
frequency component, taken in the second direction, of the 
high-frequency decomposition in the first direction of the 
2 0 first decomposing step; and 

an HH component, corresponding to the high- 
frequency component, taken in the second direction, of the 
high-frequency decomposition in the first direction of the 
first decomposing step; 
25 and further comprising: 

quantizing the UI, HL, and HH components prior to 
said communicating step. 



6. The method of claim 5, further comprising: 
30 after said quantizing step and prior to said 

communicating step, performing lossless compression on the 
LL, LH, HL, and HH components. 



7. The method of claim 5, further comprising: 
35 repeating said decomposing steps. 
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8. The method of claim 5, further comprising: 

transmitting the LL component and the quantized 
LH, HL, HH components over a communications network; 

receiving the transmitted components from the 
5 communications network; 

reconstructing a low- frequency component from the 
LL and quantized LH components using boundary-spline- 
wavelets in the second direction, and reconstructing a 
high-frequency component from the quantized HL and HH 
10 components using boundary-spline-wavelets in the second 
direction; 

then reconstructing a video image from said low- 
frequency component and said high-frequency component using 
boundary-spline-wavelets applied in the first direction; 
15 and 

displaying the results of the reconstructing 
steps on a video display. 

9. The method of claim 8, further comprising: 

20 after said quantizing step and prior to said 

communicating step, performing lossless compression on the 
LL, LH, HL, and HH components; and 

after said receiving step, performing losslesB 
decompression on the communicated LL, LH, HL and HH 

25 components. 

10. The method of claim 8, further comprising: 
repeating said decomposing steps a selected 

number of times prior to said communicating step; and 
30 repeating said reconstructing steps the selected 

number of times prior to said displaying step. 

11. The method of claim 5, further comprising; 
prior to said decomposing steps, dividing each 

35 frame into a plurality of image blocks; 
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wherein said decomposing steps are performed for each 
image block of each frame, so that each of said image 
blocks comprises a LL component and quantized LH, HL, and 
HH components. 

5 

12. The method of claim 11, further comprising: 

transmitting the LL component and the quantized 
LH, HL, HH components for each of said image blocks of each 
frame over a comnunications netvork; 
10 receiving the transmitted components from the 

communications netvork; 

for each of said image blocks, reconstructing a 
low-frequency component from the LL and quantized IH 
components using boundary-spline-wavelets in the second 
15 direction, and reconstructing a high-frequency component 
from the quantized ^HL and HH components using boundary- 
spline-wavelets in the second direction; 

then, for each of said image blocks, 
reconstructing a video image from said low-frequency 
20 component and said high-frequency component using boundary- 
spline-wavelets applied in the first direction; 

arranging the reconstmcted video image for each 
of said image blocks into a video frame; and 

displaying the video frame on a video display. 

25 

13. A method of displaying compressed video image 
data, comprising: 

receiving compressed video image data 
corresponding to a frame and storing the compressed frame 
30 in memory, said compressed video image data arranged as 
first, second, third and fourth. frequency domain 
components ; 

in a first image direction, applying boundary- 
spline-wavelets to each column of data in the compressed 
35 frame to reconstrojct a low-frequency component from the 
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first and second frequency domain components, and to 
reconstruct a high-frequency component from the third and 
fourth frequency domain components; in a second 

image direction, applying boundary-spline-wavelets to each. 
5 row of data of the low- frequency component and high- 
frequency component to reconstruct the video image frame; 

displaying the reconstructed video image frame on 
a video display. 

10 14. The method of claim 13, further comprising: 

repeating said steps of applying boundary-spline- 
wavelets in the first and second image directions a 
selected number of times. • 

15 15. The method of claim 13, wherein said steps of 

applying boundary-spline-wavelets are performed, for each 
frame, for a plurality of image blocks of said frame. 

16. The method of claim 13, further comprising: 

20 performing lossless decompression on the 

compressed video image data, prior to said applying steps. 

17. The method of claim 13, further comprising: 
after said applying steps, storing the 

25 reconstructed video image frame in an image buffer. 

18. The method of claim 17, further comprising: 
after said storing step, magnifying the image of 

said video image frame prior to said displaying step. 

30 

19. A system for communicating video image 
information, comprising: 

an input source for providing digital video image 
information arranged as frames; and 
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compressor circuitry having an input coupled to 
said input source, for decomposing each frame of digital 
video image information in a first image direction using a 
boundary-spline-wavelet into a first low-frequency 
5 component and a first high-frequency component, and for 
then further deccnposing each of said low-frequency and 
high-frequency components of each frame of digital video 
image information in a second image direction using a 
boundary-spline-wavelet into first and second pairs of low- 

10 frequency and second high-frequency components, said first 
pair being the decomposed representation of said first low- 
frequency component and said second pair corresponding to 
a decomposed representation of said first high-frequency 
component, said compressor circuitry also having an output 

15 for presenting the first and second pairs of low-frequency 
and high-frequency ccaponents. 

20. The system of claim 19, further comprising: 

a format converter circuit coupled between said 
2 0 input video source and said compressor, for converting the 
digital input video information into a portable gray-level 
format. 

21. The system of claim 20, whei-ein said input source 
2 5 provides color video information; 

and wherein said format converter circuit is for 
converting the color video information into a plurality of 
portable gray-level format representations, each 
representation corresponding to a color component. 

30 

22. The system of claim 19, wherein said compressor 
circuitry comprises: 

decomposing circuitry for performing the 
decomposing of each frame of digital video image 
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information into the first and second pairs of low- 
frequency and high-frequency components; 

quantization circuitry for quantizing the high- 
frequency ccmponent of said first pair, and for quantizing 
5 the both the low- frequency and the high-frequency component 
of the second pair; and 

lossless coopression circuitry, having an input 
coupled to said quantization circuitry, for perfoming 
lossless compression of the first and second low-frequency 
10 and high-frequency pairs, prior to presenting the 
decomposed frames of digital video image information at the 
output of the compressor circuitry. 

23. The system of claim 22, further comprising: 

15 main controller circuitry, for controlling the 

operation of the decomposing circuitry, so that the low- 
frequency component of said first pair may be repetitively 
provided to the decomposing circuitry for further 
decomposing. 

20 

24. The system of claim 19, wherein said compressor 
circuitry comprises: 

an image buffer coupled to the input of the 
compressor circuitry for storing a frame of digital video 
25 image information; and 

a digital matrix processor for performing the 
decomposing of each frame of digital video image 
information into the first and second pairs of low- 
frequency and high-frequency components using a boundary- 
30 spline-wavelet. 

25. The system of claim 24, wherein said digital 
matrix processor operates according to matrix operations 
using precalculated matrices corresponding to spline and 

35 wavelet components and stored in said compressor circuitry. 
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26. The system of claim 19, further comprising: 

a storage unit, coupled to the output of said 
compressor circuitry, for storing the decomposed frames of 
digital video image information. 

27. The system of claim 19, wherein the output of 
said compressor circuitry is coupled to a digital network 
for communication of said decomposed frames of digital 
video image information. 

28. The system of claim 27, wherein said compressor 
circuitry comprises: 

decomposing circuitry for performing the 
decomposing of each frame of digital video image 
information into the first and second pairs of low- 
frequency and high-frequency components; 

quantization circuitry for quantizing the high- 
frequency component of said first pair, and for quantizing 
the both the low- frequency and the high-frequency component 
of the second pair; and 

lossless compression circuitry, having ah input 
coupled to said quantization circuitry, for performing 
lossless compression of the first and second low-frequency 
and high-frequency pairs, prior to presenting the 
decomposed frames of digital video image information at the 
output of the compressor circuitry; 

and wherein the output of the loBsless compression 
circuitry is coupled to the digital network so that the 
decomposed frames of digital video image information are 
communicated after the lossless compression. 

29. The system of claim 19, further comprising: 

a communications network coupled to the output of 
the compressor circuitry; 
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decompressor circuitry, having an input coupled 
to said cormiunications network, for reconstructing each 
decomposed frame of digital video image information in the 
second image direction using a boundary-spline-wavelet, and 
5 for then further reconstructing each frame of decomposed 
digital video image information in the first image 
direction using a boundary-spline-wavelet; and 

a video display, for displaying the reconstructed 
frames of digital video image information. 

10 

30. The system of claim 29, wherein said compressor 
circuitry comprises: 

decomposing circuitry for performing the 
decomposing of each frame of digital video image 
15 information into the first and second pairs of low- 
freguency and high-frequency components; 

quantization circuitry for quantizing the high- 
frequency component of said first pair, and for quantizing 
the both the low-frequency and the high-frequency component 

2 0 of the second pair; and 

and wherein said decompressor circuitry comprises: 

reconstructing circuitry, for reconstructing a 
digital output video frame from the decomposed frames of 
digital video image information, by reconstructing an 
25 approximation of the first low- frequency component from the 
first low-frequency and high-frequency pair in the second 
image direction, and an approximation of the first high- 
frequency component from the second low-frequency and high- 
frequency pair in the second image direction, and by then 

3 0 reconstructing the digital output video frame from the 

approximations of the first low-frequency component and the 
first high-frequency component in the first image 
direction. 
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31. The system of_ claim 30, wherein the compressor 
circuitry further comprises: 

main controller circuitry for controlling the 
operation of the decomposing circuitry, so that the low- 
5 frequency component of said first pair may be repetitively 
provided to the decomposing circuitry for further 
decomposing. 

and wherein the decompressor circuitry further 
comprises : 

10 main controller circuitry, for controlling the 

operation of the reconstructing circuitry so that the 
result of the reconstructing may be repetitively operated 
upon by the reconstructing circuitry, depending upon the 
number of times the decomposing circuitry repetitively 

15 decomposed the digital input video frame. 

32. The system of claim 30, wherein said compressor 
circuitry further comprises: 

lossless compression circuitry, having an input 
20 coupled to said quantization circuitry, for performing 
lossless compression of the first and second low-frequency 
and high-frequency pairs, prior to presenting the 
decomposed frames of digital video image information to the 
communications network; 
25 and wherein said decompressor circuitry further 

comprises: 

lossless decompression circuitry, having an input 
coupled to the communications network for performing 
lossless decompression of the digital video image 
30 information received therefrom. 

33. A system for displaying compressed video image 
data, comprising: 

a memory for storing a plurality of compressed 
35 video frames; 
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decompressor circuitry for reconstructing each of 
said plurality of compressed frames in a first inage 
direction using a boundary-spline-wavelet, and for then 
further reconstructing each of said plurality of compressed 
5 frames in a second image direction using a boundary-spline- 
wavelet; and 

a video display, coupled to said decompressor 
circuitry, for displaying each of said plurality of 
reconstructed frames. 

10 

34. The system of claim 33, wherein said memory is 
coupled to a communications network. 

35. The system of claim 3 3, wherein said decompressor 
15 circuitry comprises: 

a digital matrix processor coupled to said 
memory, for reconstructing each of said plurality of frames 
using precalculated matrices. 

20 36. The system of claim 33, wherein each of said 

plurality of frames are stored in said memory in a form of 
the type comprising quantized first and second pairs of 
low-frequency and high-frequency components, said first 
pair of low-frequency and high-frequency components 

2 5 corresponding to the decomposition, in the first direction, 

of a low-frequency decomposition in the second direction of 
a video image frame, and said second pair of low-frequency 
and high-frequency components corresponding to the 
decomposition, in the first direction, of a high-frequency 

3 0 decomposition in the second direction of the video image 

frame. 

37. The system of claim 36, wherein each of said 
plurality of frames are stored in said memory in a fora 
35 including the lossless compression of said quantized first 
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and second pairs of low- frequency and high-frequency 
components; 

and further comprising: 

lossless decompression circuitry, coupled between 
5 said memory and said decompressor circuitry, for performing 
lossless decompression of each of said plurality of frames 
prior to the reconstruction thereof by said decompressor 
circuitry. 

10 38. The system of claim 36, wherein said first and 

second pairs of low-frequency and high-frequency components 
correspond to the results of repetitively decomposed 
components of a video image; 
and further comprising: 

15 main controller circuitry, for controlling the 

operation of the decompressing circuitry so that the result 
of the reconstructing may be repetitively operated upon by 
the decompressing circuitry according to the number of 
times that the stored frames were repetitively decomposed. 

20 

39. The system of claim 33, wherein each of said 
plurality of frames stored in said memory correspond to a 
plurality of compressed image blocks; 

and wherein said decompressor circuitry reconstmcta 

25 each of the compressed image blocks for each of said 
plurality of compressed frames in the first image direction 
using a boundary-spline-wavelet, and for then further 
reconstructing each of the compressed image blocks for said 
plurality of compressed frames in the second image 

30 direction using a boundary-splinerwavelet. 

***** 
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